Understanding load average

Load average is quoted constantly and understood rarely. It is not CPU utilisation and it has no maximum.

What it counts

The average number of processes either running or waiting to run, over one, five and fifteen minutes. On Linux it also counts processes in uninterruptible sleep — typically waiting on disk or network storage.

That last inclusion is what makes Linux load average different from other Unixes, and it is why a machine with idle CPUs can show a high load: everything is waiting on I/O.

Reading it against core count

Load must be interpreted relative to CPUs. A load of 8 on an eight-core machine means roughly one runnable process per core — fully used, nothing queuing. The same load on a two-core machine means significant queuing.

nproc
uptime
cat /proc/loadavg

The three numbers together

The trend matters more than any single value. 1-minute well above 15-minute means load is rising now. The reverse means a spike that has passed. Three similar numbers mean a steady state.

High load, idle CPU

This combination means processes are blocked, not computing. Almost always storage, sometimes network storage that has become slow or unreachable.

vmstat 1 5           # 'b' column: blocked processes
iostat -x 1 3        # %util and await per device
ps -eo state,pid,comm | awk '$1 ~ /D/'   # who is in uninterruptible sleep

Processes in state D are the ones inflating the load. If they are all touching the same mount, you have found your problem.

What to monitor instead

Load average is a useful trend and a poor alert threshold, because the meaningful value depends on core count and workload. Alert on the specific things users feel: response time, queue depth, I/O wait. Use load average to notice that something changed, not to decide what.