sipsorcery's blog
Occassional posts about VoIP, SIP, WebRTC and Bitcoin.
Occassional posts about VoIP, SIP, WebRTC and Bitcoin.
With WebRTC starting to gain momentum I have been doing a bit of work recently on integrating a custom .Net/C++ application with WebRTC capable browsers (at the time of writing Chrome and FireFox). It’s been challenging work and there are a lot of moving parts involved with WebRTC. The good thing is once that once the integration challenges have been overcome WebRTC streaming works very well and my experience so far of WebRTC video streams are a lot better than those I’ve had with SIP video soft phones.
The SIPSorcery code base now has all the components needed to develop prototype applications that can integrate with WebRTC browsers. An example of a WebRTC video test pattern can be found here. While it is a very rudimentary application that simply adds a timestamp to a static JPEG image it does demonstrate the network and encoding pipeline necessary to get media streams working between a .Net application and a WebRTC browser.
The good thing is that if you have a project that needs .Net and WebRTC you might be able to use the SIPSorcery code as your starting point and perhaps even contribute to it. To create your own WebRTC project you need to add the SIPSorcery and SIPSorceryMedia nuget packages. The SIPSorceryMedia package uses some native and C++ dll’s and does require the Visual C++ Redistributable Packages for Visual Studio 2013 to be installed on any machine that executes it. It’s also built for x64 only so won’t run on 32 bit systems (I can build it for other platforms if anyone ever has a need).
An example of how to use the WebRTC components is in the source code of the WebRTCVideoSample project referenced above. The main classes are WebRTCDaemon and WebRtcSession. The WebRtcSession class is where the ICE connection establishment is coordinated and the connection to the WebRTC browser is set up. The WebRTCDaemon does the web socket signalling, which is not part of WebRTC but does provide a handy way to get the session established, and also the VP8 encoding of the timestamped JPEG image. I’ve included the WebRtcSession code below to show the pretty small amount of code required.
using System;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using SIPSorceryMedia;
using SIPSorcery.Net;
using SIPSorcery.Sys;
using log4net;
namespace WebRTCVideoServer
{
public class WebRtcSession
{
private const int RTP_MAX_PAYLOAD = 1400;
private const int TIMESTAMP_SPACING = 3000;
private const int PAYLOAD_TYPE_ID = 100;
private const int SRTP_AUTH_KEY_LENGTH = 10;
private static ILog logger = AppState.logger;
public WebRtcPeer Peer;
public DtlsManaged DtlsContext;
public SRTPManaged SrtpContext;
public SRTPManaged SrtpReceiveContext; // Used to decrypt packets received from the remote peer.
public string CallID
{
get { return Peer.CallID; }
}
public WebRtcSession(string callID)
{
Peer = new WebRtcPeer() { CallID = callID };
}
public void DtlsPacketReceived(IceCandidate iceCandidate, byte[] buffer, IPEndPoint remoteEndPoint)
{
logger.Debug("DTLS packet received " + buffer.Length + " bytes from " + remoteEndPoint.ToString() + ".");
if (DtlsContext == null)
{
DtlsContext = new DtlsManaged();
int res = DtlsContext.Init();
logger.Debug("DtlsContext initialisation result=" + res);
}
int bytesWritten = DtlsContext.Write(buffer, buffer.Length);
if (bytesWritten != buffer.Length)
{
logger.Warn("The required number of bytes were not successfully written to the DTLS context.");
}
else
{
byte[] dtlsOutBytes = new byte[2048];
int bytesRead = DtlsContext.Read(dtlsOutBytes, dtlsOutBytes.Length);
if (bytesRead == 0)
{
logger.Debug("No bytes read from DTLS context :(.");
}
else
{
logger.Debug(bytesRead + " bytes read from DTLS context sending to " + remoteEndPoint.ToString() + ".");
iceCandidate.LocalRtpSocket.SendTo(dtlsOutBytes, 0, bytesRead, SocketFlags.None, remoteEndPoint);
//if (client.DtlsContext.IsHandshakeComplete())
if (DtlsContext.GetState() == 3)
{
logger.Debug("DTLS negotiation complete for " + remoteEndPoint.ToString() + ".");
SrtpContext = new SRTPManaged(DtlsContext, false);
SrtpReceiveContext = new SRTPManaged(DtlsContext, true);
Peer.IsDtlsNegotiationComplete = true;
iceCandidate.RemoteRtpEndPoint = remoteEndPoint;
}
}
}
}
public void MediaPacketReceived(IceCandidate iceCandidate, byte[] buffer, IPEndPoint remoteEndPoint)
{
if ((buffer[0] >= 128) && (buffer[0] <= 191))
{
//logger.Debug("A non-STUN packet was received Receiver Client.");
if (buffer[1] == 0xC8 /* RTCP SR */ || buffer[1] == 0xC9 /* RTCP RR */)
{
// RTCP packet.
//webRtcClient.LastSTUNReceiveAt = DateTime.Now;
}
else
{
// RTP packet.
//int res = peer.SrtpReceiveContext.UnprotectRTP(buffer, buffer.Length);
//if (res != 0)
//{
// logger.Warn("SRTP unprotect failed, result " + res + ".");
//}
}
}
else
{
logger.Debug("An unrecognised packet was received on the WebRTC media socket.");
}
}
public void Send(byte[] buffer)
{
try
{
Peer.LastTimestamp = (Peer.LastTimestamp == 0) ? RTSPSession.DateTimeToNptTimestamp32(DateTime.Now) : Peer.LastTimestamp + TIMESTAMP_SPACING;
for (int index = 0; index * RTP_MAX_PAYLOAD < buffer.Length; index++)
{
int offset = (index == 0) ? 0 : (index * RTP_MAX_PAYLOAD);
int payloadLength = (offset + RTP_MAX_PAYLOAD < buffer.Length) ? RTP_MAX_PAYLOAD : buffer.Length - offset;
byte[] vp8HeaderBytes = (index == 0) ? new byte[] { 0x10 } : new byte[] { 0x00 };
RTPPacket rtpPacket = new RTPPacket(payloadLength + SRTP_AUTH_KEY_LENGTH + vp8HeaderBytes.Length);
rtpPacket.Header.SyncSource = Peer.SSRC;
rtpPacket.Header.SequenceNumber = Peer.SequenceNumber++;
rtpPacket.Header.Timestamp = Peer.LastTimestamp;
rtpPacket.Header.MarkerBit = ((offset + payloadLength) >= buffer.Length) ? 1 : 0; // Set marker bit for the last packet in the frame.
rtpPacket.Header.PayloadType = PAYLOAD_TYPE_ID;
Buffer.BlockCopy(vp8HeaderBytes, 0, rtpPacket.Payload, 0, vp8HeaderBytes.Length);
Buffer.BlockCopy(buffer, offset, rtpPacket.Payload, vp8HeaderBytes.Length, payloadLength);
var rtpBuffer = rtpPacket.GetBytes();
int rtperr = SrtpContext.ProtectRTP(rtpBuffer, rtpBuffer.Length - SRTP_AUTH_KEY_LENGTH);
if (rtperr != 0)
{
logger.Warn("SRTP packet protection failed, result " + rtperr + ".");
}
else
{
var connectedIceCandidate = Peer.LocalIceCandidates.Where(y => y.RemoteRtpEndPoint != null).First();
connectedIceCandidate.LocalRtpSocket.SendTo(rtpBuffer, connectedIceCandidate.RemoteRtpEndPoint);
}
}
}
catch (Exception sendExcp)
{
// logger.Error("SendRTP exception sending to " + client.SocketAddress + ". " + sendExcp.Message);
}
}
}
}
The SIPSorceryMedia assembly utilises the Cisco libsrtp (for secure RTP), openssl (for DTLS) and libvpx (for VP8 encoding). It also makes use of Microsoft’s Media Foundation and the ffmpeg libraries for various bits and pieces.