NVSwitch
Read it in Chapter 1 · checkpoint +50 XP →
A switch chip that gives every GPU in a server full-bandwidth NVLink to every other GPU.
Without a switch, NVLink is point-to-point, so GPUs only connect to a few neighbors. NVSwitch builds a non-blocking crossbar: in an HGX H100 server, four NVSwitch chips connect all 8 GPUs all-to-all at full NVLink bandwidth.
Newer systems extend the NVLink domain beyond a server (e.g. NVL72 racks), so 72 GPUs act like one large tensor-parallel island. The rule from NVLink still applies: the bigger the NVLink domain, the larger the tensor-parallel or expert-parallel groups you can run without touching the slower scale-out network.
nvidia-smi topo -m shows NV18 (18 bonded links) between every GPU pair in an NVSwitch system.
Related
Sources & further reading
- Modal GPU Glossary — hardware and software vocabulary
- NVIDIA H100 Tensor Core GPU
Written for LazyLab in our own words; sources are cited for depth and verification.