VPN Protocols Explained

Every VPN protocol solves the same problem: how do you create a private, encrypted tunnel through the public Internet? But they solve it differently, and those differences matter.

The protocol you choose determines how secure your tunnel actually is, how fast data moves through it, whether it survives network changes, and how much pain you'll endure setting it up. There's no universal best choice—only trade-offs that fit your situation or don't.

What a VPN Protocol Actually Does

A VPN protocol handles four things: establishing the connection (the handshake), encrypting the data (so observers see gibberish), encapsulating packets (wrapping your traffic inside other traffic), and maintaining the tunnel (keeping it alive as conditions change).

The protocol's choices in each area cascade into real-world consequences. Strong encryption with slow key exchange means security but latency. Efficient encapsulation with poor roaming support means speed until you switch networks. Every protocol makes these trade-offs differently.

OpenVPN: The Reliable Workhorse

OpenVPN became the default for good reasons. It's open source—thousands of security researchers have examined the code, found vulnerabilities, and fixed them publicly. You're not trusting a company's claims about security; you're trusting mathematics and community scrutiny.

It uses SSL/TLS for encryption, the same technology that secures your banking. You can run it over TCP (works through restrictive firewalls) or UDP (faster, less overhead). It supports AES-256 encryption, certificate-based authentication, and configurations so flexible they border on overwhelming.

That flexibility is both strength and weakness. OpenVPN can do almost anything, which means configuring it properly requires understanding dozens of parameters. Pre-configured VPN services hide this complexity; running your own server exposes it fully.

Performance is good but not exceptional. OpenVPN runs in user space rather than the kernel, adding overhead. For most uses this doesn't matter. For high-bandwidth transfers or latency-sensitive applications, it might.

OpenVPN runs on everything—Windows, macOS, Linux, iOS, Android, routers. If you need one protocol that works everywhere, this is it.

WireGuard: The Clean Slate

WireGuard started from a radical premise: what if we threw away decades of accumulated complexity and built a VPN protocol using only modern cryptography, in the smallest possible codebase?

The result is about 4,000 lines of code. OpenVPN has hundreds of thousands. This isn't minimalism for aesthetics—it's minimalism for security. Less code means fewer places for bugs to hide, easier auditing, and less attack surface.

WireGuard uses only modern cryptographic primitives: ChaCha20 for encryption, Curve25519 for key exchange, BLAKE2s for hashing. No configuration options for legacy algorithms. No compatibility modes for obsolete systems. If the cryptography is broken, you upgrade WireGuard. There's no "use the old cipher for compatibility" escape hatch.

Performance is exceptional. WireGuard runs in the kernel, uses efficient cryptography, and has minimal protocol overhead. It's measurably faster than OpenVPN and typically matches or beats IKEv2.

Configuration is refreshingly simple. A WireGuard config file is short enough to read in a minute. Define your keys, define your peers, specify allowed IPs. Done.

The trade-offs: WireGuard is younger, with less deployment history than OpenVPN's decade-plus track record. It requires static IP addresses or dynamic DNS for each peer, complicating some deployments. And its design stores peer IP addresses, which creates privacy considerations that VPN providers have had to work around.

IKEv2/IPsec: The Mobile Champion

IKEv2 combined with IPsec has one killer feature: MOBIKE. When your phone switches from WiFi to cellular, most VPN connections die. IKEv2 seamlessly maintains the tunnel across network changes without dropping the connection.

This makes IKEv2/IPsec the natural choice for mobile devices that constantly move between networks. Your VPN stays up when you walk out of the office, ride the subway, or move between cell towers.

Performance is excellent. IKEv2/IPsec is built into Windows, macOS, and iOS—native implementation means no third-party software and efficient execution.

The downside: IKEv2 uses UDP ports 500 and 4500. Some restrictive networks block these ports, making the VPN unusable. OpenVPN on port 443 looks like HTTPS traffic and passes through almost any firewall; IKEv2 doesn't have that advantage.

Configuration complexity sits between OpenVPN and WireGuard. Not as overwhelming as OpenVPN's options, but proper IPsec configuration still requires understanding encryption parameters and authentication methods.

L2TP/IPsec: The Legacy Option

L2TP creates a tunnel. IPsec encrypts it. Together they form a VPN that's built into most operating systems without additional software.

That built-in support is L2TP/IPsec's main remaining advantage. You can configure it on Windows, macOS, iOS, or Android without installing anything.

The disadvantages are significant. Double encapsulation (L2TP inside IPsec) adds overhead and reduces performance. The protocol uses ports and IP protocols that firewalls and NAT devices often handle poorly. And Edward Snowden's revelations included information about intelligence agencies compromising IPsec implementations.

L2TP/IPsec is reasonable for quick setup when nothing sensitive is at stake and modern alternatives aren't available. It shouldn't be your first choice.

PPTP: Don't Use This

PPTP is easy to configure and widely supported. Those were once meaningful advantages.

They no longer matter because PPTP is broken. Not theoretically vulnerable—actively broken. Tools exist to crack PPTP encryption in minutes. The encryption (MPPE with RC4) and authentication (MS-CHAPv2) have known, exploitable weaknesses.

PPTP provides no security benefit over an unencrypted connection against any attacker willing to spend minimal effort. Use it only when absolutely no alternative exists and only for data you don't care about protecting. Better yet, don't use it at all.

SSTP: Windows-Only Firewall Bypass

SSTP tunnels VPN traffic through SSL/TLS on port 443. Since that's HTTPS traffic, it passes through virtually any firewall.

Microsoft developed SSTP and built it into Windows. On Windows systems behind restrictive firewalls, SSTP works when IKEv2 is blocked. That's its use case.

Outside Windows environments, SSTP offers little advantage. Limited cross-platform support and proprietary development (less independent security review) make it a niche choice.

Choosing Your Protocol

Start with your constraints, not features.

If you need maximum security and can handle complexity: OpenVPN. Decades of scrutiny, open source, configurable to paranoid specifications.

If you need speed and simplicity: WireGuard. Fastest option, cleanest configuration, modern cryptography. Accept the shorter track record.

If mobile devices constantly change networks: IKEv2/IPsec. MOBIKE keeps connections alive through network transitions. Built into iOS and works seamlessly.

If you're behind restrictive firewalls: OpenVPN on TCP port 443, or SSTP if you're Windows-only. Both look like HTTPS traffic.

If you need something now without installing software: IKEv2/IPsec if your network allows it, L2TP/IPsec as fallback. Both are built into major operating systems.

If someone suggests PPTP: Decline. The convenience isn't worth the nonexistent security.

The Protocol Is Half the Story

A secure protocol badly configured is insecure. An older protocol properly hardened may outperform a newer one with default settings.

Use strong encryption—AES-256 or ChaCha20. Avoid legacy ciphers even when your protocol supports them.

Authenticate properly. Certificates beat passwords. Strong pre-shared keys beat weak ones. Multi-factor authentication adds another layer.

Update religiously. VPN software vulnerabilities appear regularly. Patching matters.

Configure conservatively. When uncertain, choose security over convenience. Disable features you don't need. Turn off legacy protocol support.

The best VPN protocol is the one you configure correctly, keep updated, and actually use.