Multi-Instance GPU (MIG)
Read it in Chapter 6 · checkpoint +50 XP →
Hardware partitioning of one GPU into up to seven isolated instances, each with its own SMs, L2 slices and memory.
MIG (A100, H100 and later) splits a GPU into up to 7 instances. Each gets a dedicated slice of SMs, L2 cache and HBM, with its own memory controllers. Faults and memory pressure in one instance don't leak into another, which makes MIG the strongest of the GPU-sharing options. See Multi-tenancy & GPU sharing.
Profiles are named by compute slices and memory, e.g. on an 80 GB H100: 1g.10gb, 2g.20gb, 3g.40gb, 4g.40gb, 7g.80gb. Not every combination fits. Placement follows fixed rules about which slices can sit together.
When to use it: inference, notebooks and CI workloads that need a fraction of a GPU and must not interfere with each other. When not to: large training jobs. They want whole GPUs and NVLink, which MIG instances can't use for peer-to-peer.
Play with the partitioner below to see how profiles pack.
Related
Sources & further reading
- NVIDIA MIG User Guide
- Modal GPU Glossary — hardware and software vocabulary
Written for LazyLab in our own words; sources are cited for depth and verification.