Watts Up
The facility asks for a 20% power cut. Keep at least 85% of the throughput.
Briefingclick to collapse
Facilities is running hot and asks every tenant to cut GPU power. Your well-tuned 7B job on 2 nodes × 8 H100s draws ~675 W per GPU.
Bring the average GPU power to ≤ 550 W while keeping ≥ 85% of current tokens/s.
GPU power limits are set per GPU with sudo nvidia-smi -pl <watts> on each node. pdsh can run a command on several nodes at once. Watch the power and throughput charts as you go.
sim time 0:40energy 0.17 kWhcost $0.46carbon 0.06 kg CO₂e
Cluster2 × 8 H100 SXM 80GB · IB-NDR
node-1
job 101 · llama7b
node-2
job 101 · llama7b
0%100% util▲ throttled✕ lostclick a GPU for details
Mission: Watts Up. You are ops@login-1 with sudo.
Type `help` for commands, `cat README.md` for notes. Objectives check themselves (right panel).
ops@login-1:~$
Objectives1/2 met
- Average GPU power ≤ 550 W
- Throughput ≥ 85% of the starting tokens/s ✓
−33 pts each
Alertsall clear
No alerts. Quiet night.
Dashboardslast 40s · hover for values
Training throughput253.6k tok/s
MFU (best job)64.4%
Mean GPU utilization96%
Facility power17.2 kW
Hottest GPU77°C