September 2009

You are currently browsing the monthly archive for September 2009.

This post is aimed at anyone thinking about contributing to the sipsorcery code base. The code base is written in C# and heavily utilises Microsoft’s .Net framework. If you’re not familiar with C# or .Net that’s not necessarily a big problem. To contribute there are two main attributes you need:

  • A good understanding of SIP and familiarity with the sipsorcery or mysipswitch services,
  • A good grasp of programming fundamentals. The language is not that important but a background in any of C, C++ or Java is useful, but not essential, as they are all pretty close to C# syntax wise.
  • If you have those two attributes and want to help or are frustrated that a fix you want is not getting done fast enough then read on.

    Quicksteps

    A minimal set of steps to get you to a point where contributiions can be made are:

  • Download a C# development tool. The recommended one is free from Microsoft Visual C# 2008 Express Edition,
  • Download a zipped up copy of the latestsipsorcery source tree,
  • Open sipsorcery-coreSIPSorcery-Core.sln,
  • Dialplan functionality is in the SIPSorcery.AppServer.DialPlan project and the DialPlanScriptHelper.cs class is the most likely one of interest,
  • Upload Patch.
  • The following paragraphs go through each step in more detail.

    Submitting Code Changes

    To contribute is as simple as uploading a patch. To do that you’ll need a codeplex account after which you can upload any sipsorcery code files you modify using the Upload Patch function. When patches get uploaded I, and hopefully in the future other sipsorcery project coordinators, will have a look over the patch and if it’s ok commit it to the source tree.

    For regular contributors instead of uploading patches and requiring a coordinator to check and commit a better option is to become a project developer and commit changes directly using Subversion. Before going down that path it’s preferred that you upload a couple of patches so that we can check that you know what you are doing and aren’t going to blow anything major up.

    Building the source

    To build the source code you’ll need two things:

  • A copy of the source which can be obtained using Subversion or downloaded as a zip file from the sipsorcery project Source page,
  • A C# compiler. A command line C# compiler is available free from Microsoft but most people build their .Net projects using an IDE which includes the compiler. The advantage of the IDE is it makes all of the build tasks simple point and click operations. The most popular .Net IDE is Microsoft’s Visual Studio .Net. The current version is Visual Studio .Net 2008 and it’s the version the sipsorcery project files are formatted for (Visual Studio .Net 2010 is in beta and I haven’t tested it out as yet). The full version of Visual Studio .Net costs a couple of thousand dollars but luckily Microsoft provide free cut down versions which are pretty much feature complete except for some more advanced and less commonly used functions. The free Visual C# 2008 Express Edition is the recommended tool for anyone wanting to start out with the sipsorcery code base.
  • Once you have an IDE installed and downloaded the source tree the solution file most people will be interested in is in the sipsorcery-core directory and is SIPSorcery-Core.sln. Double clicking on that file should open the solution in your IDE and to build simply press F6. If everything goes according to plan the build will be successful and you will now be ready to start thinking about making changes.

    Making Changes

    The main thing most people will be interested in fixing or enhancing are dialplan functions. In recognition of that the dialplan functionality was placed into a separate assembly (an assembly is what a library or component is called in the .Net World) called SIPSorcery.AppServer.DialPlan. Within that project a most of the dialplan functions exposed to the sipsorcery Ruby dialplans are contained in DialPlanScriptHelper.cs. If you want to fix something in that class you need to make your change, check the project builds and then upload the file(s) you have changed on the sipsorcery site Upload Patch.

    If you hit any snags please feel free to ask for help in the comments.

    Regards,

    Aaron

    Just a quick note to the people that are attempting to use the sipsorcery service to hack SIP providers. It needs one SQL statement to delete all your accounts which are easily identifiable by your call detail records and dialplans. I’d be suprised if a new sipsorcery account can be created and configured in less than a minute. Assuming that it means nearly an hour of work creating new accounts was wiped out in 10 seconds!

    只是快速照会人民正试图利用sipsorcery服务破解园区供应商。它需要一个SQL语句来删除所有的户口,很容易被你acll详细记录和dialplans识别。我会感到惊讶,如果一个新的sipsorcery帐户可以创建并在不到一分钟的配置。假设这意味着近一个创建新帐户的工作小时的前辈在10秒!

    Regards,

    Aaron

    I’ve been doing some work this week on implementing another measure to improve the reliability of the sipsorcery dialplan processing. Specifically the measure is to cope with a call worker process becoming unresponsive and refusing to process any more dialplan executions. This issue typically gets manifested as the “Long running dialplan” log message.

    I had thought the call worker process “stalls” has been a result of a memory leak in the DLR and that it had been solved by recycling the processes when they had reached a memory allocation of 150MB. However as soon as I put the last post on this blog about stability a call worker process stalled with a memory utilisation less than 150MB.

    I suspect as with a lot of tricky software bugs there’s more than one issue here. Stopping the memory leak has definitely improved reliability but there is still something else that can cause a call worker stall and my suspicion is some kind of Ruby dialplan script is able to tie the DLR up in knots and render it incapable of processing any further script executions. Unfortunately I’ve never been able to produce such a script but then I don’t push the limits of Ruby with classes or recursive functions etc. in my own dialplans.

    The new measure implemented today is designed to cope with a call worker process stall irrespective of its memory utilisation. So the hope is now that the sipsorcery call processing process is able to cope with anything thrown at it and is 100% reliable.

    That’s not to say there are not other things that can go wrong, I’m still none the wiser as to the two incidents a few weeks ago where the Amazon EC2 instance the sipsorcery server was running on seemed to drop off the network and not respond on remote desktop or any other protocol. Thankfully apart from those two cases it hasn’t occurred again. The next reliability measure currently in progress is to have two instances running side by side so that if there is a problem with one the other one will take over.

    On a related note I have had to cut off two users in the past two weeks for inappropriate use. One for trying to brute force a provider by running a dialplan script that cycled through usernames to see if any had a password of 1234. The second one wa for blasting the sipsorcery server with a constant volume of calls. The calls weren’t forwarded to a provider since user’s providers weren’t set up properly but it resulted in a 10% rise in the base CPU utilisation which has a small but noticeable impact on other user’s calls. It also means when I’m watching the sipsorcery call activity there’s a continuous stream of log messages which is painful.

    With cases where an obvious hack attempt is being made the account will be terminated immediately. If you’re planning on scripting up some elaborate dialplan to hack SIP Providers with sipsorcery you will be wasting a lot of time on your dialplan development as your account will get removed. With the second case and similar ones I will attempt to notify the user via email and give them a chance to fix their account and will re-activate the account if they later respond.

    Aaron

    New accounts are now enabled. The performance issue regarding the database access software has been vastly improved so the plan is to leave new account creations enabled for 48 to 72 hours. It will depend on the load that the server experiences. At this point anybody desiring a new account is recommended to create it before Wednesday.

    A note to those handful of people that have had their accounts removed.

    The same thing will happen if you create a new account and use it to send malicious traffic to SIP Providers or if you bombard the SIP Sorcery server with extremely high volumes of traffic.

    (Chinese translation of same message, thanks to Google):

    A到那些已经在其帐户中删除一小撮人注意,同样的事情会发生在当你创建一个新的帐户,并用它来发送恶意流量,园区供应商,或如果你轰击的SIP法术交通服务器。

    perfmon-dbaccessfixed

    Aaron

    One thing I’ve been meaning to do for a while is a post on how to programatically connect to the SIP Sorcery provisioning service. The service is exposed over a SOAP 1.1 interface. This post provides a brief C# code example which demonstrates how to connect using WCF. At some stage I’d also like to provide a javascript based code sample using jquery or a similar library. That may motivate someone to write an alternative user interface to the Silverlight one and appease the people who dislike it for whatever reason.

    The full source code for the C# sample can be dowloaded from here.

    The SOAP standard does not include any mechanisms for authentication. There is a web service extension available that includes a mechanism in the form of WS-Security specifications. The problem is that the specification is quite bulky and the classes provided by WCF to make it easy to use are not supported by Silverlight.

    An alternative to using a SOAP authentication mechanism is to use an HTTP one such as OAuth. The HTTP approach is appealing for at least two reasons: it’s light weight and easy to implement compared to WS-Security and it can be used for different application protocols which means if/when a REST provisioning endpoint is added the same authentication mechanism can be used.

    Incorporating REST and OAuth into the SIPSorcery provisioning interface is a little way down the road and at the moment the authentication mechanism used has been designed to work easily with the Silverlight client. The existing mechanism is SOAP specific and involves two steps:

  • Step 1 – Call the Login method and if successful a token will be returned. The token is a 384 bit random number returned as a 96 byte string.
  • Step 2 – In all subsequent provisioning SOAP requests the token needs to be included in the SOAP header in an authid element.
  • Each token is only good for one hour or until it is passed to the Logout method. While the token is current it’s a critical piece of information that allows full access to a specific user’s account as such it should never be transmitted over an unencrypted connection. The SIPSorcery provisioning interface is only exposed via HTTPS which means there is no opportunity to send a token on anything but an encrypted connection.

    Below is an example of a SOAP envelope that includes the authentication token in the authid SOAP header. The SOAP request in this case is GetCustomer and the username parameter in the body is actually the username of the customer record being requested and nothing to do with authentication.

    <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
      <s:Header>
        <Action s:mustUnderstand="1" xmlns="http://schemas.microsoft.com/ws/2005/05/addressing/none">http://www.sipsorcery.com/provisioning/IProvisioningService/GetCustomer</Action>
        <authid>8c441993d8df81d42bb5a6757f4370a504d972a21c5812166a3cc36292ccb53ab1cac2750e74815707b37ddea21cdae7</authid>
      </s:Header>
      <s:Body>
        <GetCustomer xmlns="http://www.sipsorcery.com/provisioning">
          <username>username</username>
        </GetCustomer>
      </s:Body>
    </s:Envelope>
    

    With authentication out of the way on to the C# sample.

  • Step 1 – Create a new C# console application in Visual Studio
  • Step 2 – In the Solution Explorer right click on the References folder and choose Add Service Reference.

    addserviceref

  • Step 3 – Set the service address to https://www.sipsorcery.com/provisioning.svc. To work properly with the sample code below set the namespace to Provisioning.

    setserviceref

  • Step 4 – Add the classes below to your console application. These classes are required to add a custom SOAP authentication header to each request sent to the service.

    (Code sample updated 7 Feb 2010 to adjust for a new security header format.)

    public class SIPSorceryProvisioningBehavior : IEndpointBehavior {
    
        private string m_authId;
    
        public SIPSorceryProvisioningBehavior(string authId) {
            m_authId = authId;
        }
    
        public void ApplyClientBehavior(ServiceEndpoint serviceEndpoint, ClientRuntime clientRuntime) {
            clientRuntime.MessageInspectors.Add(new SIPSorceryProvisioningMessageInspector(m_authId));
        }
    
        public void ApplyDispatchBehavior(ServiceEndpoint serviceEndpoint, EndpointDispatcher endpointDispatcher) {   return; }
        public void Validate(ServiceEndpoint serviceEndpoint) { return; }
        public void AddBindingParameters(ServiceEndpoint serviceEndpoint, BindingParameterCollection bindingParameters) { return; }
    }
    
    public class SIPSorceryProvisioningMessageInspector : IClientMessageInspector {
            
        private string m_authId;
    
        public SIPSorceryProvisioningMessageInspector(string authId) {
            m_authId = authId;
        }
    
        public object BeforeSendRequest(ref Message request, IClientChannel channel) {
            request.Headers.Add(new SIPSorcerySecurityHeader(m_authId));
            return null;
        }
    
        public void AfterReceiveReply(ref Message reply, object correlationState) { }
    }
    
     public class SIPSorcerySecurityHeader : MessageHeader
        {
            private const string SECURITY_NAMESPACE = "http://www.sipsorcery.com/security";
            private const string SECURITY_HEADER_NAME = "Security";
            private const string SECURITY_PREFIX = "sssec";
            private const string AUTHID_ELEMENT_NAME = "AuthID";
    
            private static ILog logger = AppState.logger;
    
            public string AuthID;
    
            public override bool MustUnderstand { get { return true; } }
            public override string Name { get { return SECURITY_HEADER_NAME; } }
            public override string Namespace { get { return SECURITY_NAMESPACE; } }
    
            public SIPSorcerySecurityHeader(string authID)
            {
                AuthID = authID;
            }
    
            protected override void OnWriteHeaderContents(XmlDictionaryWriter writer, MessageVersion messageVersion)
            {
                writer.WriteStartElement(SECURITY_PREFIX, AUTHID_ELEMENT_NAME, SECURITY_NAMESPACE);
                writer.WriteString(AuthID);
                writer.WriteEndElement();
            }
    
            protected override void OnWriteStartHeader(XmlDictionaryWriter writer, MessageVersion messageVersion)
            {
                writer.WriteStartElement(SECURITY_PREFIX, this.Name, this.Namespace);
            }
    
            public static SIPSorcerySecurityHeader ParseHeader(OperationContext context)
            {
                try
                {
                    int headerIndex = context.IncomingMessageHeaders.FindHeader(SECURITY_HEADER_NAME, SECURITY_NAMESPACE);
                    if (headerIndex != -1)
                    {
                        XmlDictionaryReader reader = context.IncomingMessageHeaders.GetReaderAtHeader(headerIndex);
    
                        if (reader.IsStartElement(SECURITY_HEADER_NAME, SECURITY_NAMESPACE))
                        {
                            reader.ReadStartElement();
                            reader.MoveToContent();
    
                            if (reader.IsStartElement(AUTHID_ELEMENT_NAME, SECURITY_NAMESPACE))
                            {
                                string authID = reader.ReadElementContentAsString();
                                return new SIPSorcerySecurityHeader(authID);
                            }
                        }
                    }
                     return null;
                }
                catch (Exception excp)
                {
                    logger.Error("Exception SIPSorcerySecurityHeader ParseHeader. " + excp.Message);
                    throw;
                }
            }
        }
    
  • Step 5 – Now everything is ready to use the service. The code sample below shows how to do that.
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.ServiceModel;
    using System.ServiceModel.Channels;
    using System.ServiceModel.Description;
    using System.ServiceModel.Dispatcher;
    using System.Text;
    using System.Xml;
    
    namespace SIPSorcerySOAPConsole {
    
        class Program {
            static void Main(string[] args) {
                try {
                    Console.WriteLine("Starting SIP Sorcery SOAP Console");
    
                    // First step is to login and acquire an authid security token.
                    Provisioning.ProvisioningServiceClient client = new Provisioning.ProvisioningServiceClient();
                    string authID = client.Login("username", "password");
                    Console.WriteLine("authid=" + authID + ".");
    
                    // Once the security token has been acquired it needs to be set on all subsequent requests.
                    Provisioning.ProvisioningServiceClient authenticatedClient = new Provisioning.ProvisioningServiceClient();
                    authenticatedClient.ChannelFactory.Endpoint.Behaviors.Add(new SIPSorceryProvisioningBehavior(authID));
                    Customer customer = authenticatedClient.GetCustomer("username");
                    Console.WriteLine("First Name=" + customer.FirstName + ".");
                }
                catch (Exception excp) {
                    Console.WriteLine("Exception Main. " + excp.Message);
                }
                finally {
                    Console.WriteLine("finished, press any key to exit...");
                    Console.ReadLine();
                }
            }
        }
    }
    
  • If you’ve made it this far the next question you’ll have is “now that I can connect what can I do with the it?”. For the answer the best place to go is the source.

  • The interface is in IProvisioningService.
  • All but one of the classes for the returned objects are contained in the SIPSorcery.SIP.App assembly and the SIPAssets folder. The SIPAccount class for example.
  • The exception is the Customer class which is contained in the SIPSorcery.CRM assembly.
  • Finally we would request that the interface is used sensibly. It can be used to create new sipsorcery accounts (when they are enabled again) and as they are in tight supply there may be a temptation to automate their creation. At this point we do request users stick to one account each and while we are very reluctant to suspend or remove accounts and only do so as a last resort if one user’s actions have a large impact on everyone else we will do so.

    Enjoy and plese post a comment if you are successful in connecting to the provisioning service.

    Aaron

    A few people are expressing slight unhappiness at not getting a new account. If you’re one of those we’d just like you to know that it’s from necessity not spite. Below is a graph of the CPU utilisation on the sipsorcery.com server. The red line is the CPU and the top of the graph is 100%.

    sipsorcery-perfmon

    The reason for the high utilisation is a problem we have with the database access software. If/when we can get it solved CPU utilisation is expected to drop to more like 40 or 50%. At that stage we’ll open up the system for new accounts again. In the meantime the trickle for new ones is the only way to keep the service available for anyone.

    Yes we could use a bigger Amazon EC2 instance or spread the load to multiple instances but then the cost of running it starts to go above what is viable for a free service.

    Aaron

    Sorry to those that missed out yesterday we thought all those waiting would have squeezed in but it looks like there was a bit more pent up demand than anticipated. It took a few hours for all the accounts to go so it wasn’t a stampede. Assuming the sipsorcery.com server behaves itself over the next week we will open another window for 100 new accounts on Monday the 14th of September.

    This trickle approach is only planned to be temporary and once some technical issues are solved we will go back to allowing new account creations as per normal. At this stage we don’t know when that will be but the hope is for weeks rather than months.

    Aaron

    New accounts can now be created on sipsorcery.com. As mentioned in the last post only 68 new accounts will be accepted after which another temporary break will be enforced.

    Aaron

    The opportunity to create new accounts on sipsorcery.com will be available later today (8th Sep 2009). We will be accepting another approximately 50 accounts and on a first in first serve basis. The reason for the restrictions on new accounts at the moment is primarily down to load and secondly down to keeping a more controlled environment while some reliability issues are sorted out.

    Because of the limits on the number of new accounts we are now requesting that users stick to one account each, there is no advantage to having multiple accounts. Email addresses will now also need to be verified before an account is activated to stop people setting up high numbers of spurious accounts.

    I will make a brief post on this blog when new accounts are enabled.

    Aaron