VoIP Over VPN: Why Your Remote Workers' Calls Degrade and What to Do About It

The support ticket reads the same way every time: "Calls are choppy from home, but everything else is fine. Downloads are fast. Web pages load instantly. It's just the calls." And nine times out of ten, buried in the environment details, is the line that explains everything: all traffic routes through the corporate VPN.

Full-tunnel VPN policies — every packet from the laptop goes through the company gateway, no exceptions — are a defensible security posture for data. They are a terrible transport for voice. VoIP is the most latency- and jitter-sensitive traffic a remote worker generates, and the full tunnel penalizes exactly the properties voice depends on: consistent one-way delay, steady inter-packet spacing, and small packets that fit cleanly on the wire.

This post explains why the tunnel hurts voice while leaving file transfers untouched, what specifically goes wrong inside it, why split tunneling is the correct default for real-time media, and how to measure the penalty on your own network so you have a number instead of an argument.

Why voice suffers when everything else feels fine

Bulk traffic — file sync, email, web browsing — is elastic. TCP buffers it, retransmits what's lost, and stretches transfers over whatever time the network allows. Add 60 ms of latency and a download finishes imperceptibly later. Add 2% packet loss and TCP quietly resends the missing pieces. The user experiences none of it.

Voice has no such slack. A call is a stream of small packets — roughly 50 per second, one every 20 ms, each carrying 20 ms of audio — that must arrive on time, in order, and at a steady rhythm. A packet that arrives 80 ms late is as useless as one that never arrives at all, because the moment it was supposed to fill has already played. There is no retransmission, no buffering the problem away beyond a jitter buffer of a few tens of milliseconds, no second chance. The three numbers that decide call quality — latency, jitter, and packet loss — are precisely the three numbers a VPN tunnel makes worse.

So the same tunnel produces two different experiences: the file transfer that "feels fine" and the call that stutters. Both are being degraded. Only one of them can hide it.

Anatomy of the tunnel penalty

Encapsulation overhead and fragmentation

Every packet entering the tunnel gets wrapped: a new IP header, a UDP or TCP header for the tunnel transport, and the VPN protocol's own framing. WireGuard adds about 60 bytes per packet; OpenVPN and IPsec, depending on configuration, add 40 to 90. For a G.711 voice packet of roughly 200 bytes, that's a 20–30% overhead tax on every single packet — wasteful, but survivable.

The sharper problem is MTU. A packet that was comfortably under the 1,500-byte Ethernet limit before encapsulation can exceed it afterward. Voice media packets are small enough to escape this, but signaling is not: SIP INVITEs with large SDP bodies, TLS handshakes, and video packets routinely run near the MTU. When encapsulation pushes them over, the packet either fragments — and fragments get dropped or mishandled by middleboxes constantly — or gets discarded with an ICMP message that half the internet's firewalls block. The symptoms are calls that set up slowly, video that fails while audio works, or connections that die mid-negotiation. Tunnels that don't clamp MSS or lower the tunnel MTU correctly produce these failures chronically.

TCP-based VPNs and the TCP-over-TCP meltdown

Some VPNs — OpenVPN in its common fallback configuration, SSL-VPN portals, anything tunneling over port 443/TCP to slip through restrictive firewalls — carry all traffic inside a TCP connection. For real-time media this is close to the worst possible design.

RTP tolerates loss by design: a lost voice packet is simply skipped, the jitter buffer conceals a few milliseconds, and the call moves on. But when RTP rides inside a TCP tunnel, the tunnel doesn't know that. TCP guarantees delivery, so when a tunnel packet is lost, TCP stalls the entire tunnel until the retransmission arrives. Every voice packet queued behind the loss — packets that arrived at the tunnel perfectly on time — now waits for a retransmit round trip. One lost packet becomes a 100–300 ms stall for the whole stream. Under even 1% loss on the path, a TCP tunnel converts a mild, concealable problem into rolling bursts of delay that no jitter buffer can absorb.

This is why UDP-based tunnel transports behave categorically better for voice. WireGuard is UDP-only. IPsec ESP is effectively UDP. DTLS-based VPNs (like AnyConnect's DTLS channel) exist specifically so real-time traffic isn't strangled by TCP semantics. If your VPN client shows a "TCP fallback" state, your voice quality problem may be one checkbox deep.

The detour through the concentrator

Here is the cost that dwarfs the others: the path. A full tunnel doesn't send your packets toward their destination — it sends them toward the VPN gateway first, wherever that is.

A remote worker in Denver calling into a meeting hosted in a Denver-area cloud region might have a 15 ms direct path. If the company's VPN concentrator is in a New Jersey data center, every voice packet now flies to New Jersey, gets decrypted, exits to the internet, travels to the meeting server, and the return traffic makes the same trip in reverse. That hairpin can turn 15 ms into 90 ms before a single packet is queued. For international workers tunneling to a headquarters concentrator on another continent, one-way delays blow past the 150 ms threshold where conversations start talking over each other — the point at which callers stop having a conversation and start having an exchange of monologues.

And the concentrator itself is a shared chokepoint. Every remote worker's Windows updates, cloud backups, and video streams converge on the same appliance. Voice packets queue behind that bulk traffic in the concentrator's buffers, and any QoS marking your traffic carried is typically ignored or stripped inside the tunnel — the encrypted wrapper hides the inner DSCP values from every router along the way. Queuing delay at a loaded concentrator is bursty by nature, and bursty delay is the literal definition of jitter.

Encryption jitter on loaded endpoints

Encryption cost is small — single-digit milliseconds on modern hardware — but it is not constant. On a laptop running at high CPU (a compile, a video render, forty Chrome tabs) or an overloaded concentrator, per-packet crypto time varies with scheduling pressure. Voice doesn't care that the average added delay is 3 ms; it cares that packet 1,001 took 1 ms and packet 1,002 took 25 ms. Variance is jitter, and jitter accumulates from every stage of the tunnel: encryption at the client, queuing at the concentrator, decryption at the gateway. Each stage's variance stacks on the last.

Split tunneling: the right default for real-time media

The fix is not a faster VPN. It's routing policy: send real-time media directly to its destination and keep corporate data traffic in the tunnel. This is split tunneling, and for voice and meeting traffic it should be the default, not the exception.

Every major UCaaS and meeting vendor publishes the IP ranges and UDP port ranges their media uses precisely so IT can exclude them from the tunnel. Microsoft explicitly recommends split tunneling Teams media — its optimize-category endpoints — and we cover the Teams-specific configuration in detail in Microsoft Teams call quality and your network. Zoom, Webex, RingCentral, and hosted PBX providers publish equivalent lists. The implementation is a route table entry or a VPN client policy: traffic to these ranges goes out the local internet connection; everything else stays tunneled.

The security counter-argument, addressed honestly

The objection writes itself: "we tunnel everything so it's all encrypted and inspected." For voice media, the objection is weaker than it sounds.

VoIP media is already encrypted end to end. WebRTC-based services — Teams, Zoom, Meet, and every browser-based platform — mandate SRTP for media, keyed via DTLS, with signaling over TLS. Desk phones and softphones registering to modern hosted platforms use TLS and SRTP as standard. Wrapping SRTP in a VPN adds a second layer of encryption to traffic that was never plaintext, at the cost of everything described above. Nor does the inspection argument hold: your security stack cannot inspect SRTP payloads inside the tunnel anyway — it just sees encrypted media at higher latency.

What split tunneling actually changes is that the media's destination IPs are visible to the local network instead of hidden inside the tunnel — a modest metadata exposure, mitigated by splitting only the published, well-known service ranges of your sanctioned providers rather than "all UDP." Corporate file shares, internal apps, and admin interfaces stay in the tunnel where they belong. You are not weakening the perimeter; you are declining to route already-secure traffic through a chokepoint that damages it.

If policy or regulation genuinely forbids any traffic outside the tunnel, then invest in making the tunnel voice-capable: UDP transport (WireGuard or IPsec, never TCP fallback), concentrators placed near your workforce and your meeting provider's regions, capacity headroom, and MTU clamping. That is the expensive road to a worse result than split tunneling, but it is at least a road.

Measure the penalty: the A/B test

Every argument above can be replaced with one number, measured on your own network in five minutes.

Run the free VoIP quality test twice from the same machine, minutes apart: once with the VPN connected, once with it disconnected. The test sends real encoded audio over an actual WebRTC media path and reports latency, jitter, packet loss, and a MOS score for each run. Diff the results.

A typical full-tunnel comparison looks like this: direct, 28 ms latency, 4 ms jitter, 0% loss, MOS 4.3; through the tunnel, 96 ms latency, 22 ms jitter, 0.8% loss, MOS 3.4. That delta — nearly a full MOS point — is the difference between calls nobody thinks about and calls that generate tickets. It is also the entire business case for split tunneling, expressed in a format that survives a meeting with the security team: not "the VPN feels slow," but "the tunnel costs us 0.9 MOS and triples our jitter, and here are both reports."

Have several remote workers run the same A/B pair. If the penalty is consistent across users, the concentrator path is the problem. If only some users suffer, their local networks deserve a look first — our home office VoIP checklist covers the usual suspects, from Wi-Fi placement to bufferbloat.

What IT should monitor going forward

Split tunneling is a fix, not a finish line. Remote voice quality drifts as ISPs change routing, home networks degrade, and VPN configurations get "temporarily" changed and never changed back. A sustainable posture looks like this:

• Baseline every remote worker. One test with the VPN up and one with it down, recorded at onboarding. Without a baseline, every future complaint starts from zero.
• Re-test after VPN client updates. Client upgrades silently re-enable full tunnel or flip UDP transports to TCP fallback more often than vendors admit. A two-minute A/B test after each rollout catches the regression before the tickets do.
• Watch jitter and loss, not just latency. Latency is stable and easy to alarm on; jitter and loss are where degradation shows first. Alert when a user's jitter trends above roughly 20 ms or loss above 1% — the thresholds where concealment stops working.
• Track MOS over time per user. A slow slide from 4.3 to 3.8 over a month is a failing cable modem, a saturated concentrator, or an ISP routing change — all invisible in a single snapshot, obvious in a trend line.

For the broader playbook — QoS at home, ISP selection, escalation paths, and managing quality across a distributed team — see our guide to VoIP quality for remote teams. And when the next "calls are bad from home" ticket lands, make the first response a link to the test, run twice: VPN on, VPN off. The diagnosis is usually sitting in the diff.