The CUDA software stack

Read it in Chapter 2 · checkpoint +50 XP →

Kernel driver → user-mode driver → CUDA runtime → libraries → frameworks. Version mismatches live at every seam.

From the bottom up:

1. Kernel module (nvidia.ko and friends). It owns the hardware, memory management (including Unified Memory / UVM), interrupts and error reporting. NVIDIA publishes an open-source version, open-gpu-kernel-modules, the default for Turing and newer GPUs. The Xid messages you read in dmesg come from here.
2. User-mode driver (libcuda.so): the CUDA driver API, shipped with the driver.
3. CUDA runtime and toolkit (libcudart, nvcc): what applications compile against.
4. Libraries: cuBLAS, cuDNN, NCCL, CUTLASS.
5. Frameworks: PyTorch, JAX, plus higher-level tooling.

Compatibility rule: a newer driver runs older CUDA runtimes, but not the reverse. nvidia-smi prints the driver version and the maximum CUDA version it supports. Containers bundle their own runtime but must use the host's driver, injected by the NVIDIA Container Toolkit. See Containers & GPUs.

Related

Sources & further reading

• NVIDIA open-gpu-kernel-modules — open-source Linux GPU kernel driver
• CUDA C++ Programming Guide
• NVIDIA Container Toolkit

Written for LazyLab in our own words; sources are cited for depth and verification.