Roofline model

Read it in Chapter 5 · checkpoint +50 XP →

Attainable FLOP/s = min(peak compute, arithmetic intensity × memory bandwidth). Tells you which wall you're hitting.

Every kernel has an arithmetic intensity (AI): FLOPs performed per byte moved from memory. The roofline says:

attainable FLOP/s = min( peak FLOP/s ,  AI × memory bandwidth )

Plot it on log-log axes and you get a slanted "memory roof" meeting a flat "compute roof" at the ridge point. For an H100 in BF16 that's ~989 TFLOP/s ÷ 3.35 TB/s ≈ 295 FLOPs/byte.

• Elementwise ops (add, GELU, layer norm) have AI around 1. They're hopelessly memory-bound, so fuse them.
• Large matmuls have AI in the thousands. They're compute-bound, and that's where tensor cores shine.
• Attention without fusion is memory-bound. FlashAttention restructures it to stay on-chip.

The PRACE GPGPU guide uses the same reasoning for HPC kernels: first decide which roof you're under, then optimize for that resource.

Related

Sources & further reading

• Roofline model (Williams, Waterman & Patterson, 2009)
• PRACE Best Practice Guide – GPGPU — GPU programming and tuning practice for HPC
• Modal GPU Glossary — hardware and software vocabulary

Written for LazyLab in our own words; sources are cited for depth and verification.