Impactful scheduling for GPU clusters

Ai2 manages thousands of NVIDIA H100, B200, and B300 GPUs in clusters of 88 to 1024 GPUs, supporting about 150 internal researchers across diverse AI domains like LLM/VLM training and robotics RL. To optimize scheduling and prioritize high-impact research while maintaining full occupancy, Ai2 introduced a “scheduling contract.” Workloads declare a minimum runtime for protected progress, after which they can be rebalanced. Alternatively, workloads with zero minimum runtime are subject to preemption but are not charged to any budget.

This report details Ai2's novel "scheduling contract" approach to GPU cluster management, unlike typical systems that prioritize only utilization or throughput.