OSPF cost and the traffic taking the wrong path

A frequent and avoidable problem: a site has a fast primary link and a slow backup, and traffic uses the backup. Nothing is broken, and OSPF is doing precisely what it was configured to do.

How cost is derived

By default, OSPF cost is a reference bandwidth divided by interface bandwidth. The default reference is 100 Mbps. That was reasonable when it was chosen and is now the source of the problem.

Cost is an integer with a minimum of 1. At the default reference, anything at 100 Mbps or faster gets cost 1. A 100 Mbps link, a 1 Gbps link and a 10 Gbps link are indistinguishable. OSPF cannot prefer the fast one because, as far as it is concerned, they are equal.

The fix

Raise the reference bandwidth so modern speeds produce distinct costs:

router ospf 1
 auto-cost reference-bandwidth 100000    # in Mbps, so 100 Gbps

Now 1 Gbps costs 100, 10 Gbps costs 10, and 100 Mbps costs 1000. The ordering is meaningful again.

The important caveat

This must be set identically on every router in the OSPF domain. Mixed reference bandwidths produce inconsistent path selection, and the resulting routing is difficult to reason about because each router computes a different view of the same topology. Change it in one maintenance window, everywhere.

Verifying

show ip ospf interface brief      # cost per interface
show ip route ospf                # what was actually chosen
traceroute destination            # what the traffic really does

Manual cost

Individual interfaces can be set with ip ospf cost, which is useful for expressing preference that bandwidth does not capture — a metered link, or a satellite path where latency matters more than capacity. Use it deliberately and record why, because an unexplained manual cost is a puzzle for whoever comes next.