New Accounts Disabled

Unfortunately it had to happen at some point, the single dedicated server that sipsorcery is running on is overloaded and I have been forced to disable new accounts. In actual fact I waited too long and the service is really starting to creak a bit with call set up times sometimes taking longer than they should and registrations occasionally being dropped because the responses don’t get processed in time.

At this point there is no plan to increase the server capacity to allow new accounts to be created. If the server utilisation drops off because some existing users drop off then I will be able to allow some new accounts but apart from that at this point it’s a regretful sorry to any new users.

  1. Korb’s avatar

    That is a shame, hopefully current users will donate to expand this great service to those who are yet to experience it. Thanks for your generosity and hard work.

    Reply

  2. Avi Marcus’s avatar

    Here’s an idea: It’s not just how many people there are, but what they are doing, and how optimized those processes are.

    I’m not sure how much of the server capacity this takes up, but there are 2-3 things that re-occur (or not) on a schedule:
    Incoming registrations, pings for those registrations, and outgoing registrations.
    If they are a significant in their server load, perhaps we should optimize them?
    Most ATA’s are probably fine with 3600 second registration time, with the SS ping, a ping from the ATA, or port forwarding.
    So that said, is pinging from the ATA better than SS sending pings?
    And outgoing registration, other than Spikko, I haven’t heard anyone say that under 3600 was much of a problem, but many registrations may be below that.

    Hmm.. dial plan execution time, perhaps there’s a way to optimize that? I’ve heard that ruby on rails doesn’t scale well, is ruby the same way?

    p.s. how many users/incoming & outgoing registrations/and call set-ups per second are there? I’m intensely curious 🙂

    Reply

    1. sipsorcery’s avatar

      3rd party registrations are what chews the most CPU. There are approx 10,500 of them. I’ve already been through a couple of optimisation cycles with the registration agent which isn’t to say more can’t be done but it’s now likely to be diminishing returns for the time I put into it.

      I’ve no idea how many of the 3rd party registrations are being used and I’ve started thinking about some way of getting active users to confirm they are still being used so that inactive ones can be disabled.

      Reply

      1. jadam’s avatar

        If you were commercial too many users would be a nice problem to have, some “guidance” could actually be useful. Providers like sipgate.co.uk and localphone seem to enforce a max reg time of 600, so a value of 3600 is ignored. I’m using voxalot to get voicemail and need registration to voxalot to receive MWI but voxalot accept 3600. I’ll try 3600 in the ATA now. The alternatives would be something like just having SS as a trunk in pbxes.com who are at 99.5% for unpaid accounts lol

        Reply

      2. Avi Marcus’s avatar

        Yikes, localphone and sipgate.co.uk have a max of 600 seconds?
        If they are a large amounts of the registrations, it might make a difference to be in touch with them to ask them to raise that limit. Not necessarily that they’d listen, though…

        Reply

      3. Mike Telis’s avatar

        Many people are using SS as a SIP gateway to Google Voice. Most (if not all) of them receive incoming calls and callbacks from IPKall, IPCOMMS, Gizmo or Sipgate. I don’t know about Sipgate, but the other three can forward incoming calls to SS (that is, people don’t need to register if they want to receive incoming calls from these DIDs).

        Could you check how many of those 10,500 registrations are to IPKall, IPCOMMS, Gizmo and Sipgate? (assuming Sipgate also supports forwarding to SIP)

        If it’s a quite big number, you could probably prohibit registrations to these SIP proxies and write a note that the users should configure their services to forward incoming calls to SS.

        Reply

        1. The Chosen One’s avatar

          I’m fairly sure that’s what I’m doing. I have my IPkall account set to ring SIP Sorcery, not the other way around. I don’t even think IPKall allows people to register to it’s servers anymore.

          Reply

        2. sipsorcery’s avatar

          40% are sipgate, 25% are Gizmo, no other providers are more than 4%. I can’t see anywhere on the sipgate site to forward to a SIP URI so I’d say registrations are required for it.

          I could contact sipgate about lifting the registration interval but there would be a risk that they would simply block sipsorcery. Although with 4k active registrations there’s no doubt they are already aware of it. With gizmo I could prevent registrations from sipsorcery and instead get users to set up a SIP URI forward at the gizmo end but that would then be another extra step in what’s already a pretty complicated process, given most sipsorcery users these days are interested in configuring Google Voice calls.

          Reply

          1. Avi Marcus’s avatar

            25% are gizmo…
            “but that would then be another extra step” well, as of now they have to add it as a provider. If you set up forwarding, then you don’t need a provider entry at all. So it would be the same number of steps. (unless they want to use gizmo directly for 800 calls.)

            Of interest, would it help to force gizmo registration to 3600 seconds, or are they set to 3600 anyway? That could be an intermediate solution…

            Reply

          2. sipsorcery’s avatar

            There were about 150 Gizmo accounts out of the 2200 that were using a registration expiry of less than 3600. Since it’s an easy thing to do I’ve bumped those 1500 up to 3600.

            Reply

          3. Mike Telis’s avatar

            > With gizmo I could prevent registrations from sipsorcery and instead get
            > users to set up a SIP URI forward at the gizmo end but that would then be
            > another extra step in what’s already a pretty complicated process, given
            > most sipsorcery users these days are interested in configuring Google
            > Voice calls.

            See, either you do that or disable new registrations for ever. I don’t think the chance of people abandoning their SS accounts is very high. More likely, they’ll start adding new SIP accounts for their friends thus increasing the load (while the number of SS accounts will remain the same).

            It would be more fair if you pushed existing users a bit rather than turn back on new users. Besides, new accounts @Gizmo are disabled, so it will be 1-step instructions for *existing* users.

            Reply

          4. Avi Marcus’s avatar

            ipkall let’s you register, afaik.
            Ipcomms let’s you register to sipconnect.ipcomms.net and has not-cheap US termination. So if folks are registering to there, it should be changed.
            Gizmo5 lets you register for incoming and outgoing, but since you can set forwarding – which may have better call quality, and certainly less overhead, registering via cron is unnecessary.
            And sipgate.. I don’t see a way to do sip forwarding. But folks may have it registering too frequently. I don’t use them, gizmo5 has been my go-to – one less PSTN bounce for me!

            Reply

            1. jadam’s avatar

              Found that sipgate peers with some providers like iptel and sipphone, see
              http://www.sipgate.de/user/news.php (find 000477 in the page)
              and
              http://lists.iptel.org/pipermail/services/2010-January/000097.html
              so to dial an iptel user I can use
              000477from sipgate.co.uk.
              Unfortunately this number is not accepted as valid for forwarding in the sipgate.co.uk webpage, but possibly might work in sipgate.com
              Would be a bit longwinded, sipgate forward to iptel, who then forward to SS.

              Reply

            2. Avi Marcus’s avatar

              erm, meant to say ipkall doesn’t let you register.

              Reply

              1. Mike Telis’s avatar

                I remember that IPKalls didn’t allow you to register but I heard they changed it some time back. Anyway, without statistics we’re just speculating. All it takes is to sort registrations by domain to find the most popular. If some of them offer forwarding to SIP, prohibit registrations and explain the users that they should forward the account@sip.sipsorcery.com.

                Reply

              2. capmin’s avatar

                Just by coincidence, I learned about SS’s new accounts policy from a post on the Google Voice Tips & Tricks forum. I then discovered that SS has been asking for donations and was surprised at how few people have given. I value SS a lot and will contribute ASAP, although I hate to do so through Paypal 🙁 So I have a couple of suggestions/questions for Aaron. Couldn’t you email all registered users asking them to donate? Could you get the word out at sites like broadbandreports.com, dslreports.com, and the Google Voice forum–or have others do it for you? Or, maybe it’s time to charge an annual subscription for the SS service? I’m sure all these things have occurred to you, but what might be less obvious is how much active users appreciate SS and would be willing to pay for it. Maybe it’s as simple as informing people that the hardware is getting “creaky” and a small donation would ensure SS’s reliability (and get this message out beyond the SS website itself).

                Reply

              3. natek’s avatar

                I don’t see why one would need to register gizmo accounts since you can use sip to forward and make calls…you don’t even need it registered to make 800 calls. One can directly dial sip URL using sys.Dial(“number@proxy01.sipphone.com”)

                It would be interesting to know why people are registering gizmo as providers…I am sure it’s because they were trying it when long long time ago gizmo was having issue with google voice and/or using some guide which says to do it that way.

                Reply

                1. natek’s avatar

                  forgot to say all of the above is assuming everyone is not using gizmo to make paid calls…but I would thing that would be very small number of people if any

                  Reply

                2. Yitzchok’s avatar

                  Can you please provide some stats on the system like how many registrations, invites (cps) etc… it would be interesting to see how well sipsorcery scales.

                  Is it possible to run a profiler on the code to see where is the (slow) code?

                  Reply

                  1. sipsorcery’s avatar

                    The biggest job the server performs is 3rd party registrations which are currently running at 70 registrations per second (not including retransmits & authentications). Calls per second rarely go above 10 simultaneous, the blue line on the status graph actually shows sample points on simultaneous calls.

                    I’ve already run a memory and performance profiler over the code a few times at different points in fact. The “hot spot” is communicatiing with the database. Currently the database is MySQL which means the MySQL ADO.Net driver is being used however up until recently the database was SQL Azure which used the Microsoft SQLClient ADO.Net driver. Both drivers put the system under similar load which means there’s not a lot more performance that can be squeezed out of that layer.

                    Reply

                    1. Avi Marcus’s avatar

                      This a radical suggestion that would probably take much time..
                      I recall a game called Antrophia that ran on MySQL. To get the most out of performance – the game made calculations every second and updates lots of info – the creator, Solatis, wrote a wrapper that basically ran the entire DB in memory. It stored ONE file with a stream of the updates to the DB to be ran on system crash/reboot to get the stored setting back upto date. Somehow, he passed the more complex queries back to the DB (I don’t know how that was kept up to date, though…)
                      I’m not sure how much the relevant data would take up in memory (it may be too large), but I have heard of using this approach…

                      Reply

                    2. Yitzchok’s avatar

                      What part of the system is hitting the database so much that the database becomes a bottleneck?

                      Maybe introducing caching into the system would help (memcached).

                      Reply

                    3. sipsorcery’s avatar

                      The thing that hits the database hardest is the registration agent. I’ve already gone over the code and de-normalised the database schema to increase the efficiency. That’s not to say more can’t be done but it’s at a stage where the returns are diminishing and the effort increasing. Note that the database is not a bottleneck, far from it the peak CPU utilisation I have obereved for mysqld is 4%, rather it’s the overhead that goes along with consuming the database service from the sipsorcery .Net application. That overhead is tiny but when you start getting into operation volumes over 100 per second it starts to become noticeable and eventually critical.

                      The reason I haven’t used memcached or another caching mechanism is that I’ve always viewed it as critical that the data supplied by the sipsorcery web service is up to date. If there was caching involved the SIP Provider bindings panel in the Silverlight client would always be displaying old data.

                      Reply

                    4. Mike Telis’s avatar

                      > I’ve always viewed it as critical that the data supplied
                      > by the sipsorcery web service is up to date.

                      Hmm, could you service Silverlight requests at sip.sipsorcery.com? I have a feeling it would be cheaper (resource-wise) than running without memcache…

                      Reply

                    5. Yitzchok’s avatar

                      There should be a way to delete an item from the cache you can do this after a user updated the data in the silverlight app.

                      Do you think using a NoSQL database like MongoDB or RavenDB would help speed this up?

                      Reply

                    6. sipsorcery’s avatar

                      @Mike: Yes it would be more performant to run everything including the web site from one server. However it would be a short term fix and would increase the pain when the server capacity again gets exhausted. Along with the fact that there’s been 2 years of effort put into making the service scale across multiple servers and bringing everything back to a single server would be going backwards.

                      @Yitzchok: It’s not just deletes that the Silverlight client does it also shows in real-time whether a certain provider is registered or not. Troubleshooting incoming calls from providers is a common function and it would be difficult to do so without being able to determine whether the information in the Silverlight client was accurate or not.

                      I have been wondering the same thing about a NoSQL database. I did put quite a bit of effort into making sipsorcery work with Amazon’s SimpleDB and in fact had it working but the latency on queries and the fact that it’s not guaranteed to be consistent meant I didn’t end up pulling the trigger. And Microsoft’s SQL Azure came along at the same time and it seemed like the perfect solution at the time. A NoSQL database would only be an improvement if the client side plumbing was less than an SQL database, the issue is in the client not the database. NoSQL databases are something on my list to look into but who knows when I’ll get to it.

                      Reply

                    7. Micah’s avatar

                      Hi admin. Very impressive work you’ve done until now, and I hope it can make you some money, at the same time.

                      What upgrade will you need for SS to reopen for new accounts? How much would that cost? Is that the $220/mo you are requesting for the dedicated server? Or will it be more? Is there a way that an individual (or group of individuals) could pay for some new accounts, or that would be too little too late?

                      Reply

                      1. sipsorcery’s avatar

                        To open up for free accounts we’d need another server. The cost for that is 2.5k per year not counting that the existing server has only had 1.5k donated towards it.

                        There is some capacity to milk from the existing server by tweaking the code here and there and seeing if there are any inactive accounts that can be disabled. The auction mechanism seems like a good and fair mechanism to allow a trickle of new accounts onto the existing server and if there is any surplus from the auctions that will go towards a new server.

                        If the whole eBay auction thing proves to be useful then there’s no reason not to put some accounts on as BuyNow so that anyone who wants to get up and running immediately has such an option. It’ll just take a week or so to see how it plays out and then get the administrative side of things sorted out.

                        Reply

                      2. Sam’s avatar

                        When will there be more new accounts available?

                        Reply

Reply

Your email address will not be published. Required fields are marked *