MFU (Model FLOPs Utilization)
Read it in Chapter 5 · checkpoint +50 XP →
Useful model FLOPs per second divided by the hardware's peak. The honest headline number for training efficiency.
For dense transformers, training costs about 6 × parameters × tokens FLOPs (2 forward + 4 backward). MFU compares what you achieved to the theoretical peak:
MFU = (6 · N · tokens_per_second) / (num_GPUs · peak_FLOP/s)
It deliberately ignores recomputation from Activation checkpointing (that's HFU, hardware FLOPs utilization), so it only rewards useful work.
What's good? 35–55% is typical for well-tuned large-model training on H100-class clusters. Above 60% is excellent.
What drags it down: small micro-batches, exposed communication (no overlap), pipeline bubbles, data-loader stalls, Stragglers, throttled clocks and plain FP32. The Feed the Beast mission walks through these in order.
Use it in a mission
Related
Sources & further reading
- PaLM: MFU definition (Chowdhery et al., 2022)
- Distributed Training (T. Tsugriy) — parallelism strategies, collectives, memory
Written for LazyLab in our own words; sources are cited for depth and verification.