TCP Options: MSS, Window Scaling, and More

TCP's designers knew they couldn't predict the future. They didn't know about gigabit Ethernet, satellite links, or data centers where machines sit meters apart but need to move terabytes. So they built an escape hatch: TCP options. These optional header fields let endpoints negotiate capabilities beyond the original specification, adapting a protocol designed for ARPANET to networks its creators never imagined.

The Options Field

TCP options live in the header, between the standard fields and the payload. Each option declares its type and length, and the field can hold up to 40 bytes total. The critical negotiations happen during the three-way handshake—both sides advertise what they support, agree on common ground, and commit to those parameters for the connection's lifetime.

If one side doesn't recognize an option, it ignores it. This simple rule is what makes TCP evolvable. New capabilities can be deployed incrementally; old implementations don't break, they just don't benefit.

Maximum Segment Size (MSS)

MSS answers a practical question: how big can a TCP segment be before it causes problems?

The problem is IP fragmentation. When a packet exceeds the Maximum Transmission Unit of a network link, routers chop it into pieces. This hurts performance—if any fragment is lost, the entire packet must be retransmitted. Worse, many firewalls drop fragments entirely, treating them as suspicious.

MSS prevents this by advertising the largest segment you're willing to receive. The calculation is straightforward: take your interface's MTU (typically 1500 bytes for Ethernet), subtract 20 bytes for the IP header and 20 for TCP, and you get 1460 bytes. Each endpoint advertises its MSS during the handshake, and each direction can differ—you respect what the receiver tells you.

Path MTU Discovery extends this by probing the entire path. When a router encounters an oversized packet, it sends back an ICMP "Fragmentation Needed" message. The sender shrinks its segments and tries again. This matters for paths that traverse VPN tunnels or PPPoE connections, where the actual MTU is smaller than Ethernet's standard 1500 bytes.

Window Scaling

Here's the original sin of TCP: the receive window field is 16 bits. Maximum value: 65,535 bytes.

In 1981, that seemed generous. Today it's a chokepoint.

The bandwidth-delay product determines how much data can be "in flight"—sent but not yet acknowledged. A transatlantic connection with 100ms round-trip time and 1 Gbps bandwidth could theoretically sustain 12.5 megabytes in flight. But with a 64KB window? You send 64KB, then wait. Send 64KB, wait. A 64-kilobyte window on a gigabit transatlantic link is like trying to fill a swimming pool through a straw—you could pump faster, but you're waiting for confirmation that each drop arrived.

Window Scaling fixes this with a multiplier. During the handshake, each side advertises a shift count from 0 to 14. The window value gets left-shifted by that amount. A shift of 7 multiplies by 128, allowing windows up to 8 megabytes. A shift of 14 allows over a gigabyte.

This must be negotiated during the initial handshake—you can't enable it later. Both sides must support it. But once enabled, TCP can finally saturate high-bandwidth, high-latency links.

TCP Timestamps

Timestamps do two things, and the second one is stranger than you'd expect.

First, better round-trip time measurement. Without timestamps, TCP estimates RTT from ACK timing, but this gets ambiguous during retransmissions—did the ACK respond to the original segment or the retransmit? With timestamps, every segment carries a sender timestamp and an echo of the receiver's last timestamp. RTT calculation becomes unambiguous.

Second, protection against sequence number wraparound. TCP sequence numbers are 32 bits—they wrap after 4 gigabytes. On a 10 Gbps connection, you burn through the entire sequence space in under four seconds. Without timestamps, TCP would be hallucinating—mistaking ghosts of old packets for new data. The timestamp acts as a secondary identifier, letting TCP distinguish between a legitimate segment and a delayed duplicate from a previous wraparound.

Selective Acknowledgment (SACK)

Traditional TCP acknowledgments are cumulative: "I've received everything up to byte 5000." This creates a problem when packets arrive out of order or with gaps.

Imagine segments 1, 2, 4, and 5 arrive, but segment 3 is lost. The receiver can only acknowledge through segment 2. The sender knows something's wrong, but what? Did segments 3, 4, and 5 all vanish? Just segment 3? Without more information, the sender either retransmits everything (wasteful) or waits and probes (slow).

SACK lets the receiver say exactly what it has: "I've received 1-2 cumulatively, and I also have 4-5." The sender retransmits only segment 3. Recovery from multiple losses becomes surgical rather than speculative.

SACK requires negotiation with the SACK-Permitted option during the handshake. Once enabled, the receiver can include SACK blocks specifying up to three or four non-contiguous ranges. On lossy networks—wireless, congested paths, anything with packet loss—SACK is the difference between acceptable performance and misery.

The Negotiation Dance

Options negotiation follows a strict choreography:

1. SYN: Client advertises supported options with its preferred parameters (MSS, Window Scale shift count, SACK-Permitted, Timestamps)
2. SYN-ACK: Server responds with its own parameters for shared options
3. ACK: Confirms the negotiated parameters

Some options can differ by direction—each side has its own MSS. Others require mutual support—Window Scaling and SACK only activate if both sides agree. Timestamps are flexible; technically one side can use them alone, but both typically enable them together.

Once the handshake completes, the parameters are locked. You cannot add options mid-connection. This makes the initial negotiation critical—and it's why middleboxes that strip options (some firewalls do this) cause subtle, maddening performance problems.

Key Takeaways

• TCP options are the escape hatch that lets a 1970s protocol survive in a gigabit world—backward compatible, incrementally deployable, negotiated per-connection.
• MSS prevents IP fragmentation by advertising maximum segment size; Path MTU Discovery extends this to probe the entire network path.
• Window Scaling breaks the 64KB barrier, enabling high throughput on high-latency links by allowing megabytes of data in flight.
• Timestamps improve RTT measurement and prevent sequence number wraparound—essential when connections can burn through 4GB of sequence space in seconds.
• SACK transforms loss recovery from guesswork to precision, telling senders exactly which segments to retransmit.
• Options are negotiated once, during the handshake, and locked for the connection's lifetime—making that initial exchange quietly critical to performance.