Language Models

When Speed Kills Stability: Demystifying RL Collapse from the Training-Inference Mismatch
The relentless push for faster inference creates a dangerous training-inference mismatch that silently kills RL with LLMs. We reveal the vicious cycle—particularly acute in reasoning and agentic RL—and introduce masked importance sampling (MIS), sequence-level importance sampling that masks rollouts outside the trust region.
Jiacai Liu, Yingru Li, Yuqian Fu, Jiawei Wang, Qian Liu, Yu Shen
Sep 17, 2025 1 min read Research, Theory