MTU and the packets that vanish

There is a specific failure that teaches everyone the same lesson eventually: logging in works, listing a directory works, and downloading a file hangs forever. No errors anywhere. The cause is almost always MTU.

What is happening

Every link has a maximum frame size. Ethernet is usually 1500 bytes. Put a tunnel over it and the tunnel header eats into that, so the usable payload shrinks. WireGuard typically leaves 1420. PPPoE leaves 1492. GRE, IPsec and VXLAN each take their own slice.

A sender that does not know this sends a full-size packet with the Do Not Fragment bit set. The device that cannot forward it is supposed to send back an ICMP “fragmentation needed” message. If that message is blocked, and it very often is, the sender learns nothing. It retransmits the same oversized packet forever. Small packets sail through. Large ones disappear silently.

Finding the real MTU

ping -M do -s 1472 8.8.8.8     # 1472 + 28 = 1500
ping -M do -s 1392 8.8.8.8     # 1392 + 28 = 1420

Work down until it succeeds. Add 28 for the IP and ICMP headers to get the path MTU.

Fixing it

Three options, in order of preference. Set the correct MTU on the interface if you control both ends. Clamp TCP MSS on the router, which makes TCP negotiate a size that fits. Or stop blocking ICMP type 3 code 4, which is what lets path MTU discovery work as designed.

iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
    -j TCPMSS --clamp-mss-to-pmtu

The habit worth forming

Whenever a tunnel is involved and the symptom is “works for small things, hangs for large”, check MTU before anything else. It is not an exotic cause. It is the most common one, and it is invisible in every log you will look at.