Fair-share scheduling

Read it in Chapter 2 · checkpoint +50 XP →

Scheduler policies that balance access between teams based on what they're entitled to and what they've recently used.

On a shared cluster, "first come, first served" lets one team's 1,000 queued jobs starve everyone else. Fair-share scheduling (e.g. Slurm's multifactor priority) lowers the priority of users and accounts that recently used more than their share, with usage decaying over time.

The ingredients

• Shares: each account's entitlement, often matching funding or team size.
• Usage decay: a half-life (e.g. 7 days) so past usage is forgiven gradually.
• QOS limits: caps on concurrent GPUs, wall time or job count, so no single job blocks the cluster.
• Preemption: low-priority or opportunistic jobs can be checkpointed and requeued when priority work arrives. This only works if jobs checkpoint well.

Fairness is also about transparency: publish usage per team, queue wait times and energy, so allocation debates use data instead of anecdotes.

Related

Sources & further reading

• Slurm documentation

Written for LazyLab in our own words; sources are cited for depth and verification.