SecuritySecurity track
Multi-tenancy & GPU sharing
Read it in Chapter 6 · checkpoint +50 XP →
Time-slicing, MPS and MIG: three ways to share a GPU, with very different isolation guarantees.
Shared clusters must decide how tenants share GPUs:
| Approach | Isolation | Good for |
|---|---|---|
| Whole-GPU allocation (scheduler-enforced) | Strong | Training, anything serious |
| MIG instances | Strong (hardware partitioned memory and SMs) | Inference, notebooks, CI |
| MPS (Multi-Process Service) | Weak: shared address-space faults can affect others | Cooperative jobs from one team |
| Time-slicing | None for memory; context switches | Dev and test only |
Enforcement matters more than policy. The scheduler should be the only path to a GPU: cgroup device controls restrict /dev/nvidia* to processes inside a job allocation. Without that, any user with a shell on a node can quietly use idle GPUs. See Cryptojacking on GPU clusters.
Pair it with least-privilege accounts, no shared service users, audit logs and per-tenant accounting so usage is visible and fair (see Fair-share scheduling).
Use it in a mission
Security
Night Shift
A GPU is busy with no job scheduled. Investigate, contain and remove persistence.
Related
Sources & further reading
Written for LazyLab in our own words; sources are cited for depth and verification.