Authentication and Security in SIP

SIP Signaling From the Wire Up: Part 8 of 12

SIP was designed in an era when the internet was a more trusting place. The protocol's authentication mechanisms are functional but not particularly strong by modern standards, and its default mode of operation sends everything in plain text. This isn't a secret or a flaw that nobody has noticed. It's a known characteristic of the protocol that the industry has addressed through a combination of transport layer security, application level protections, and operational practices.

Understanding how SIP handles authentication and what the security landscape looks like puts you in a position to evaluate the security posture of your own VoIP deployment and to understand why certain security recommendations exist.

Digest authentication

The most common authentication mechanism in SIP is digest authentication, which we saw briefly in the registration post. Let's look at it more closely.

Digest authentication is a challenge/response mechanism. The server challenges the client to prove it knows a shared secret (the password) without transmitting the password itself. Here's the complete flow with the relevant headers.

The client sends a request (a REGISTER, an INVITE, or any other method the server requires authentication for):

REGISTER sip:example.com SIP/2.0
...

The server rejects it with a 401 Unauthorized and includes a challenge:

SIP/2.0 401 Unauthorized
WWW-Authenticate: Digest realm="example.com",
  nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093",
  qop="auth",
  algorithm=MD5,
  opaque="5ccc069c403ebaf9f0171e9517f40e41"

The key fields in the challenge are:

realm is the authentication domain. This is typically the SIP domain and is included in the hash calculation.

nonce is a unique value generated by the server for this specific challenge. The nonce prevents replay attacks because each challenge produces a different nonce, and the correct response changes even if the password stays the same.

qop stands for quality of protection. When set to "auth," it means the server wants authentication. When "auth-int," it also wants integrity protection of the message body.

algorithm specifies the hash algorithm to use. MD5 is the most common by far. SHA-256 is specified in RFC 8760 but real world support is still limited.

The client computes the digest response by combining the username, password, realm, nonce, method, and URI using the specified algorithm, then sends the request again with an Authorization header:

REGISTER sip:example.com SIP/2.0
Authorization: Digest username="alice",
  realm="example.com",
  nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093",
  uri="sip:example.com",
  response="6629fae49393a05397450978507c4ef1",
  algorithm=MD5,
  qop=auth,
  nc=00000001,
  cnonce="0a4f113b",
  opaque="5ccc069c403ebaf9f0171e9517f40e41"

The response field is the computed hash. The nc (nonce count) increments with each use of the same nonce, preventing replay within a nonce's lifetime. The cnonce (client nonce) is a value generated by the client that adds additional randomness to the hash calculation.

The server performs the same computation using the password it has stored for alice. If the computed hash matches the response value in the Authorization header, the client has proven it knows the password.

The weakness of digest authentication

Digest authentication is better than sending a password in plain text, but it has known weaknesses.

The most significant is that it relies on MD5, which is considered cryptographically weak. While it's not trivial to reverse an MD5 hash to recover the original password, it is feasible to brute force short or common passwords by computing the hash for every candidate password and comparing against the captured response. Tools exist specifically for cracking SIP digest hashes from captured REGISTER or INVITE messages.

This means that if someone can capture your SIP traffic (which is straightforward on a shared network since SIP typically runs unencrypted over UDP), they can extract the digest challenge and response, and then attempt to crack the password offline. Short passwords, dictionary words, and predictable patterns are especially vulnerable.

The practical takeaway is that SIP credentials should be long, random, and unique. Treat them like any other system password. The digest mechanism provides a thin layer of protection, but it's not something to rely on against a determined attacker.

Transport security: UDP, TCP, and TLS

By default, SIP messages travel in plain text over UDP. Anyone who can capture network traffic on the path between your phone and the server can read every SIP message, including the INVITE headers that reveal who's calling whom, and the SDP bodies that reveal the IP addresses and ports where audio will flow.

SIP supports three transport protocols:

UDP is the most common. It's connectionless, lightweight, and fast. The downside is no encryption, no delivery guarantee (SIP handles retransmissions itself), and a practical size limitation: RFC 3261 specifies that SIP messages larger than 1300 bytes should be sent over TCP instead of UDP to proactively avoid IP fragmentation. (The actual IP fragmentation point on standard Ethernet is around 1472 bytes of UDP payload, but the RFC sets a lower threshold to provide a safety margin across varying path MTUs.)

TCP provides reliable, ordered delivery. It solves the fragmentation problem and is sometimes required for large SIP messages. But like UDP, TCP SIP is unencrypted.

TLS (Transport Layer Security) encrypts the SIP messages in transit. When SIP runs over TLS, it's sometimes referred to as SIPS (though the terminology is used inconsistently). TLS protects the signaling from eavesdropping and tampering between the phone and the next hop. A connection secured with TLS looks like this in SIP:

Via: SIP/2.0/TLS 10.0.0.50:5061;branch=z9hG4bK74bf9

Note the transport is TLS and the port is 5061, which is the standard port for SIP over TLS (versus 5060 for unencrypted SIP).

Important caveat: TLS encrypts the signaling only. It protects the SIP INVITE, the SDP, the headers, and the registration credentials from being read in transit. It does not encrypt the audio. For audio encryption, you need SRTP.

Also, TLS is typically hop by hop. If your phone connects to your PBX over TLS, but your PBX connects to the SIP trunk provider over UDP, the signaling is encrypted on the first leg and unencrypted on the second. End to end encryption of signaling requires TLS on every hop, which is often not the case in practice.

SRTP: encrypting the audio

SRTP (Secure Real-time Transport Protocol) is the encrypted version of RTP. When SRTP is negotiated in the SDP, the audio stream between the two endpoints is encrypted, protecting the voice conversation from eavesdropping.

SRTP negotiation appears in SDP as a change to the media line:

m=audio 49170 RTP/SAVP 0
a=crypto:1 AES_CM_128_HMAC_SHA1_80
  inline:d0RmdmcmVCspeEc3QGZiNWpVLFJhQX1cfHAwJSoj

The RTP/SAVP profile (instead of RTP/AVP) indicates SRTP. The a=crypto attribute contains the encryption suite and the key material.

Some implementations use DTLS-SRTP instead, where the encryption keys are negotiated through a DTLS handshake rather than being included in the SDP. This approach is used in WebRTC and is generally considered more secure because the key material isn't transmitted in a signaling message that might pass through intermediaries.

The practical state of SRTP adoption varies. Most hosted PBX providers support it. Many on premises PBX systems support it. But in a lot of deployments, it's not enabled by default, and the administrator has to specifically turn it on. If you're evaluating the security of a VoIP deployment, check whether SRTP is configured, not just available.

SIP toll fraud

Toll fraud is probably the most tangible security threat to SIP systems, and it's one that costs real money. The basic scenario is: an attacker gains access to your SIP system and uses it to make expensive international or premium rate calls. The calls are billed to your account. You discover the problem when you get a phone bill for thousands of dollars.

The most common attack vector is brute forcing registration credentials. The attacker scans the internet for SIP devices listening on port 5060 (which is easy to find because SIP registration traffic is constant and predictable). Once they find a SIP server or PBX, they try registering with common usernames and passwords. Extensions like "100," "200," or "admin" with passwords like "1234," "password," or the same string as the extension are shockingly common and are the first things attackers try.

What a brute force attempt looks like in a trace is a rapid series of REGISTER messages with different credentials, each one getting a 401 challenge or 403 rejection. Automated tools can try thousands of combinations quickly.

Once the attacker successfully registers, they send INVITEs to international premium rate numbers (often in regions where the attacker has arranged to receive a per minute payment from the terminating carrier). The calls look legitimate to your SIP trunk provider because they're coming from an authenticated registration on your system. The charges accumulate rapidly, sometimes reaching tens of thousands of dollars over a weekend.

Protecting against toll fraud

Strong, random passwords. Every extension should have a unique, random password of at least 12 characters. Never use the extension number as the password. Never use dictionary words.

Rate limiting and fail2ban. Configure your PBX to detect and block IP addresses that are generating a high volume of failed authentication attempts. Tools like fail2ban can monitor SIP logs and automatically firewall offending IPs. Asterisk, FreeSWITCH, and most modern PBX platforms support this either natively or through add-ons.

Geographic call restrictions. If your business doesn't need to call international numbers, block them on your SIP trunk. Most trunk providers allow you to set geographic restrictions on outbound dialing. If you only call domestic numbers, there's no reason to leave international dialing enabled.

Restrict registration by IP. If your phones are on known IP addresses or subnets, configure your PBX to only accept registrations from those addresses. This is simple and effective.

Monitor your call records. Set up alerts for unusual call patterns: a spike in international calls, calls outside business hours, calls to premium rate numbers, or a single extension placing an unusual volume of calls. Early detection limits the financial damage.

Use TLS for registration. Encrypted registration prevents attackers from capturing and cracking credentials off the wire. It doesn't prevent brute force attacks (the attacker can still try credentials against the TLS port), but it eliminates one attack vector.

The security reality

Most SIP deployments in the real world are not fully secured. Internal traffic runs over UDP without encryption. Digest authentication with MD5 is the norm. SRTP is available but often not enabled. SIP ALG on the router is mangling packets in ways that complicate security analysis.

This is not ideal, but it's the reality that most VoIP systems operate in. The practical approach is to prioritize the protections that address the most likely threats: strong credentials to prevent toll fraud, fail2ban to block brute force attempts, geographic restrictions to limit the damage of a compromise, and TLS on externally facing connections to protect signaling from eavesdropping on untrusted networks. Full end to end encryption of both signaling and media is achievable but requires deliberate configuration across every component in the path.

Next up: Call Features in SIP: Hold, Transfer, and Forwarding, where we look at how the buttons on your phone translate into SIP messages on the wire.