HardwareArchitecture track
Streaming Multiprocessor (SM)
Read it in Chapter 1 · checkpoint +50 XP →
The GPU's core building block: schedulers, registers, shared memory, CUDA cores and tensor cores.
An SM is roughly what a "core" is on a CPU, but built for throughput. Each SM has warp schedulers, a large register file, configurable shared memory / L1 and execution units: FP32/INT "CUDA cores", special-function units and tensor cores.
A kernel launch is split into thread blocks. Each block lands on one SM and stays there. The SM interleaves the warps of all resident blocks every cycle, so when one warp waits on memory another issues.
Why you care as an operator
- SM count × clock sets peak FLOP/s. A GPU throttled to 55% clock (thermal or power) delivers ~55% of the math. See Power & thermals.
DCGM_FI_PROF_SM_ACTIVEandSM_OCCUPANCY(from DCGM) tell you whether SMs are actually busy. "GPU utilization" in nvidia-smi only means some kernel was running, not that every SM was.
Related
Sources & further reading
- Modal GPU Glossary — hardware and software vocabulary
- CUDA C++ Programming Guide
- PRACE Best Practice Guide – GPGPU — GPU programming and tuning practice for HPC
Written for LazyLab in our own words; sources are cited for depth and verification.