Memory coalescing

Read it in Chapter 4 · checkpoint +50 XP →

When a warp's 32 threads read adjacent addresses, the hardware merges them into a few wide transactions.

Global memory is accessed in aligned segments (32-byte sectors, grouped into 128-byte lines). If the 32 threads of a Warp & SIMT read 32 consecutive 4-byte floats, the hardware serves them with one 128-byte transaction. If they read with a large stride, it may need 32 separate transactions, a 32× bandwidth penalty.

Classic fixes from GPU programming practice (e.g. the PRACE guide)

• Structure-of-arrays instead of array-of-structures.
• Have thread i touch element i, not i × stride.
• Stage awkward access patterns (transposes, stencils) through shared memory, then write out coalesced.
• Watch for shared-memory bank conflicts when you do.

Profilers (Nsight Compute) report "sectors per request". Ideal is 4 for 4-byte loads.

Related

Sources & further reading

• PRACE Best Practice Guide – GPGPU — GPU programming and tuning practice for HPC
• CUDA C++ Best Practices Guide

Written for LazyLab in our own words; sources are cited for depth and verification.