February 2010

You are currently browsing the monthly archive for February 2010.

I had a question on the forums about creating a PC based widget to display the callerID on an incoming call by maintaining a persistent connection to the sipsorcery telnet monitoring server. One problem with the design is that sipsorcery no longer uses telnet the end of telnet for sipsorcery, so the design would need to change to use SSH instead. The question mentioned using Adobe’s AIR to build the application and while I don’t know a lot about AIR it’s likely that a telnet client would be easy to implement, since it’s not much more than a TCP socket, but SSH would be a lot harder as there’s a lot of extra work to negotiate and set up the encrypted channel.

As it happens at the same time the SSH change was made I changed the way the Siverlight client retrieves the notifications messages from the server from using a telnet connection to a HTTPS one. Part of the motivation was exactly the same as the problem in the previous paragraph, it was going to be tricky to implement an SSH client in Silverlight. Another reason was that the Silverlight client can only establish TCP connections to destination ports 4302 to 4332 and I had begun to find that a bit frustrating, I couldn’t use the console from a previous workplace, some internet cafes etc.

The HTTPS service that the Silverlight client uses to get notifications is . The “pull” at the end of the service URL is pertinent and the notifications mechanism requires that clients poll the sipsorcery web server to pull notifications down. Such a pull mechanism is not ideal as it means unneccessary traffic and load but after spending the best part of two weeks fighting with Microsoft’s PollingDuplexHttpBinding only to conclude that it’s completely broken with IIS6 and switch back to a traditional pull mechansim. Maybe at some point I’ll revisit it, the sipsorcery web site has since moved to Windows Azure which uses IIS7 and on another front HTML5 has introduced Web Sockets which achieve the same as the PollingDuplexHttpBinding.

The ability to pull notifications from the sipsorcery web server is something that can be used right now to build an application like the callerID widget. The Silverlight client consumes the service using a WCF SOAP endpoint. Connecting to WCF SOAP endpoints is ok if you’re going to be writing a client in .Net (C#, VB.Net etc) but it can get tricky from non .Net languages especially when the service involves authorisations which the sipsorcery one does. One great thing about WCF is that with very little effort a service endpoint can be added to support an alternative interface. In this case REST and JSON (I’m not deliberately trying to set a record for acronyms) are a lot more universally understood compared to SOAP and .Net’s XML serialisation. I’ve spent a few hours adding the REST and JSON interface to the notifications service and have deployed it to the sipsorcery Windows Azure web site, the URL is https://www.sipsorcery.com/notificationspull.svc/rest/. The service interface is:

[ServiceContract(Namespace = "http://www.sipsorcery.com/notifications/pull")]
    public interface INotifications
    {
        [OperationContract]
        [WebGet(UriTemplate = "isalive", ResponseFormat = WebMessageFormat.Json)]
        bool IsAlive();

        [OperationContract]
        [WebGet(UriTemplate = "login?username={username}&password={password}", ResponseFormat = WebMessageFormat.Json)]
        string Login(string username, string password);

        [OperationContract]
        [WebGet(UriTemplate = "logout")]
        void Logout();

        [OperationContract]
        [WebGet(UriTemplate = "getpollperiod", ResponseFormat = WebMessageFormat.Json)]
        int GetPollPeriod();

        [OperationContract]
        [WebGet(UriTemplate = "subscribeforaddress?subject={subject}&filter={filter}&addressid={addressid}", ResponseFormat = WebMessageFormat.Json)]
        string SubscribeForAddress(string subject, string filter, string addressID);
        
        [OperationContract]
        [WebGet(UriTemplate = "getnotificationsforaddress?addressid={addressid}", ResponseFormat = WebMessageFormat.Json)]
        Dictionary<string, list=""><string>> GetNotificationsForAddress(string addressID);

        [OperationContract]
        [WebGet(UriTemplate = "closeconnectionforaddress?addressid={addressid}")]
        void CloseConnectionForAddress(string addressID);
    }

I’ve put together a Ruby sample that hooks up to the service and pulls down the notifications. I’ve put the sample in github so that it can be updated if needs be.

require 'httpclient'
require 'json'
require 'UUIDTools'
 
puts "sipsorcery get notifications sample"
 
notificationsURL = "https://www.sipsorcery.com/notificationspull.svc/rest/"
myUsername = "username"
myPassword = "password"
addressID = UUIDTools::UUID.random_create
filter = "event%2053"
 
client = HTTPClient.new
resp = client.get_content("#{notificationsURL}login?username=#{myUsername}&password=#{myPassword}")
authID = resp.delete('"')
puts "authID=#{authID}"
 
resp = client.get_content("#{notificationsURL}subscribeforaddress?subject=console&filter=#{filter}&addressid=#{addressID}", nil, "authID" => authID)
puts "Notifications subscribe response=#{resp}"
 
30.times do
  resp = client.get_content("#{notificationsURL}getnotificationsforaddress?addressid=#{addressID}", nil, "authID" => authID)
  if !resp.empty?
    notifications = JSON.parse(resp.to_s)
    notifications[0]["Value"].each do |notification|
      puts notification.chomp
    end
  end
  sleep(1)
end
 
# Close the notifications connection.
client.get_content("#{notificationsURL}closeconnectionforaddress?addressid=#{addressID}", nil, "authID" => authID)
 
puts "finished"

I will explain each of the service methods in my next post.

At the moment I’m unable to properly connect to the sipsorcery services or any other Amazon sites I have tested including www.amazon.com. I’m situated on the bottom of Australia (literally next stop Antarctica, the ice breakers sail past my house on the way out) so whatever network issue is causing the problem is hopefully not affecting too many people elsewhere around the globe. From the looks of the forums no one else has noticed an issue as normally there’s a big spike in activity whenever sipsorcery has a problem.

On a ping test to the sipsorcery SIP servers the packet loss is about 95% which is pretty weird but does indicate it’s probably a routing glitch somewhere rather than a cable cut. Hopefully it will clear itself up in a few hours. There seems to have been a few fun and games around in the last 24 hours WordPress downtime and I wouldn’t be surprised if the issue I’m experiencing has cascaded down from whatever caused that.

Voxeo announced an initiative last week for voice biometrics which means being able to authenticate a caller based on their voice.

I gave it a spin using PerSay and after sorting out a tiny problem with the VXML referencing the wrong IP address the demo application worked exactly as specified. If anyone wants to try it out it can be reached on sip:9991445422@sip.voxeo.net

when /^234$/ then sys.Dial("sip:9991445422@sip.voxeo.net") # PerSay voice biometrics application.

The flow of the application is:

  • You will be asked to enter an ID using your phone’s keypad (don’t use 123456 as that’s the ID I used),
  • After the ID has been entered the application requests you to speak 0 to 9 three times to complete the enrollment process,
  • After the enrollment is complete hangup and then redial and this time after entering your ID you will be asked to speak 0 to 9 once and the PerSay server will authenticate your voice.

Ideally I’d like to have a voice biometrics application that didn’t need a numeric ID and could authenticate based on arbitrary conversation. That would make the process quicker and more flexible and open up applications such as being able to translate calls on the fly and associate each caller to a known identity.

This morning I stumbled across a hosted Asterisk service that’s being offered for free by a company called Aretta. To sign-up for the free service called, NetPBX Free, you need to visit this link https://www.aretta.com/free/. The free service doesn’t seem to be advertised on the main Aretta site so it looks like they are using word of mouth to let people know about they offering.

As far as using the service goes the account set up was painless and I was up and running with a dedicated Asterisk instance in about 5 minutes. However after that things weren’t quite so easy. The NetPBX Free admin interface is fairly comprehensive and also includes a FreePBX install, which is a web management portal that sits on top of Asterisk. Anyone not very familiar with Asterisk is going to really struggle as the configuration options are bamboozling and the FreePBX interface means there are two ways to get most things done. That’s not a deficiency of the NetPBX product that’s just how Asterisk is. Anyone who thinks sipsorcery is difficult to get a grasp on would be in for a rude awakening with Asterisk. Even the basics of setting up a SIP account and configuring the dialplan to allow inbound and outbound calls can be a tricky exercise.

The main problems I had with the Aretta NetPBX product was the lack of visibility to the Asterisk console, which is absolutely essential with such a complex piece of software, and likewise the lack of visibility to the configuration files. There is a Logging tab available that dumps the Asterisk log file to a web page but it seems to be using a cron job or something as it only refreshes every 10 or 15 minutes. As far as the system configuration files go I wanted to set up an extension to allow my default SIP account to dial into music on hold. After setting up the dialplan I dialled into the extension but the call failed with a declined error response. The likely cause of the failure was a problem with the music on hold set up but it’s very hard to diagnose that sort of thing with no console and no access to the musiconhold.conf file. The FreePBX interface does have a music on hold section and after fiddling around in there for a while and doing a few restarts I was able to get it kind of working but it took a lot longer than it would have normally.

To be fair to Aretta it’s a free service and SSH access is available on the paid versions. The NetPBX product may be good for someone already familiar with Asterisk and with lots of time and patience on their hands to play around with but I wouldn’t recommend it for someone wanting to learn Asterisk. A better option for that would be to use one of Voxilla’s EC2 AMI’s and fire up an instance. I’ve used the Voxilla images a few times to check something on Asterisk or when I wanted to do some tests with a media server and sipsorcery.

The sipsorcery web site is now running from the Windows Azure cloud. Previously the web site was running from one of the EC2 SIP server instances which wasn’t ideal as if that SIP server had an issue it meant it was not possible to access the web site.

There are no configuration changes required to use the Silverlight client on the web site, it will still be accessible on the same URL’s sipsorcery.com or www.sipsorcery.com. However if anyone had the previous sipsorcery.com redirect to https://www.sipsorcery.com/sipsorcery.html bookmarked they will need to modify it to http://www.sipsorcery.com/. If anyone is wondering why the page is no longer available over https it’s because it makes it neater for Windows Azure to have two applications, one that hosts the web services on an https end point and one that deals up static pages on an http end pont. In the new deployment even though the Silverlight client is accessed from an http endpoint it will still use the web services over https so is secure as far as web service communications from the client to the server.

A change is required for anyone using the call management service which was previously available on http and https end points and is now only available on an https endpoint. The URL for web callbacks is now https://www.sipsorcery.com/callmanager.svc/webcallback?user=username&number=1234567.

It’s always been on my todo list to spend more time with Ruby with a big motivation being able to write some more powerful sipsorcery dialplans. Last week I got an incentive to finally do it when I came across a Ruby on Rails cloud platform called Heroku. Heroku operates on top of Amazon’s EC2 but rather than dealing with EC2 instances and images they abstract that all away and allow programmers to work at an application layer. The Heroku experience is close in some respects to Microsoft’s Windows Azure cloud platform, which I’ve also been spending a fair bit of time with. There’s pros and cons to each of course, Heroku’s is a lot more polished, a lot quicker to pickup and a lot quicker to deploy with while Windows Azure is more powerful given that the .Net framework is more comprehensive and has deeper hooks into the underlying Windows OS than the Ruby on Rails software does. In fact it would probably be a better comparison to compare an ASP.Net MVC web application hosted on Windows Azure to Heroku’s offering. The same pros and cons listed previously still apply and for a programmer unfamiliar with either Heroku and Ruby on Rails have a much smaller learning curve but in my opinion an ASP.Net MVC application is more powerful and manageable.

What’s all this got to do with sipsorcery?

As part of my exploration of Heroku & RoR I thought it would be worthwhile to get a basic web app deployed that would hook into the sipsorcery web services. It would give my exercise of attempting to learn RoR a goal and maybe it could be the starting point for another programmer out there to put together a non-Silverlight interface for sipsorcery, something which comes up regularly on the sipsorcery forums.

The sipsorcery web services have always been available via a SOAP API although as far as I know nothing except my Silverlight client has ever consumed it. SOAP APIs are out of fashion these days though and REST and JSON are now the flavour of the month, they are also easier to use so that’s good news. I have exposed a few sipsorcery methods over REST, specifically the login and get SIP account methods, and it was there that I wanted to hook up from a Heroku RoR app.

The result of my efforts is at https://sipsorcery.heroku.com/auth/login. It’s a completely bare bones application, absolutely no effort has gone into the user interface, that lists the ID’s, usernames and domains of the first 10 SIP accounts belonging to a sipsorcery user. I’ve only been using RoR for a week so I’m not fully confident about the application being fully secure but the two communications channels, browser to the Heroku app and the Heroku app to the sipsorcery REST API are both SSL, so I’m satisfied enough to use my own sipsorcery username and password with it.

If there are any RoR programmers out there that use sipsorcery and are interested in a non-Silverlight interface my very crude code for the above application is hosted on github.

The main concept of the RoR app is the connection to the sipsorcery REST API. The Ruby code to accomplish that is very succinct and I’ve posted it below for anyone interested. The user and pass values need to be replaced with valid sipsorcery account details and if the code works it will dump SIP account details as a block of JSON encoded data which looks very messy but is easy for a programmer to work with.

require 'httpclient'
require 'rexml/document'

puts "starting json test..."

baseURL = "https://www.sipsorcery.com/provisioning.svc/rest/"

client = HTTPClient.new
resp = client.get_content("#{baseURL}customer/login?username=user&password=pass")
authID = REXML::Document.new(resp).elements[1].text
puts "authID=#{authID}"

sipClient = HTTPClient.new
resp = sipClient.get_content("#{baseURL}sipaccounts?count=10", nil, "authID" => authID)
puts "SIP Accounts=#{resp}"

I’ve just uploaded an alpha release candidate of a new local version of sipsorcery (local indicates it’s designed to run on a PC and is suitable for a small number of users).

The new version can be downloaded from codeplex.

It’s best to post any questions or issues with the release candidate on the dedicated forum.

I plan to put together some release notes, update the documentation and incorporate any feedback that may come up and will then make another release.

There have been 5 outages of sip1.sipsorcery.com in the last 24 hours which is verging on the ridiculous. I’ve even swapped IP addresses between sip1 and sip2 to see if the issue was constrained to one physical instance but that didn’t work.

I’m not getting much love from Amazon support Access outages seem to be caused by traffic volumes so at this point I’m not hopeful for smooth operation of sipsorcery.

My hypothesis is that an Amazon network element is blacklisting the sipsorcery server due to its traffic profile of lots of small UDP packets or something. The evidence being that only the 174.129.236.7 server ever fails and it’s the one with 80% of the traffic. Also from the server logs I can see that the server doesn’t crash and the sipsorcery servers are keep running trying to talk to the outside World it’s just that no network traffic can get in and out.

The worry is that once the primary server fails the traffic levels will build up on the secondary server and it will get blacklisted as well.