3

Semantic-Aware Scheduling for GPU Clusters with Large Language Models

SchedMate brings semantic context to GPU-cluster scheduling by extracting information from source code, runtime logs, and historical jobs. Its LLM-based scheduling advisor, metric tracker, and failure handler enhance existing schedulers through a non-intrusive interface.

Zerui Wang, Qinghao Hu, Ana Klimovic, Tianwei Zhang, Yonggang Wen, Peng Sun, Dahua Lin

RL in the Wild: Characterizing RLVR Training in LLM Deployment

This study characterizes reinforcement learning with verifiable rewards in a deployed LLM training environment. It analyzes workload variation, GPU idling, parallelism, data management, and load imbalance, and introduces the PolyTrace benchmark suite for evaluation with realistic workloads.

Jiecheng Zhou, Qinghao Hu, Yuyang Jin, Zerui Wang, Peng Sun, Yuzhe Gu, Wenwei Zhang, Mingshu Zhai, Xingcheng Zhang, Weiming Zhang

Efficient Training of Large Language Models on Distributed Infrastructures: A Survey

This survey reviews systems for efficient large language model training on distributed infrastructure. It covers accelerators, networks, storage, scheduling, parallelism, computation, communication, memory optimization, and reliability during long training runs.

Jiangfei Duan, Shuo Zhang, Zerui Wang, Lijuan Jiang, Wenwen Qu, Qinghao Hu, Guoteng Wang, Qizhen Weng, Hang Yan, Xingcheng Zhang, Xipeng Qiu, Dahua Lin, Yonggang Wen, Xin Jin, Tianwei Zhang, Peng Sun