Yingru (Richard) Li
I make reinforcement learning on frontier models stable, efficient, and scalable.
I am a core contributor to RL fundamentals at xAI (now SpaceX Super Intelligence), contributing to the Grok RL big runs and to one of the longest stable RL runs to date. Before that I led RL stability and efficiency research for LLM reasoning and agents at ByteDance.
Ph.D. in Computer and Information Engineering, The Chinese University of Hong Kong (2025), advised by Prof. Zhi-Quan (Tom) Luo, with Prof. Benjamin Van Roy (Stanford & DeepMind) on my thesis committee.
Highlights
Why LLM-RL collapses, and the fix
We traced collapse to the training–inference mismatch and introduced masked importance sampling (MIS), made principled as Trust Region Masking (TRM). Adopted, shipped, or discussed by:
Open-source RL infrastructure
Authored the Rollout Correction framework for general off-policy training in veRL (20k+ GitHub stars), and contributed its variance-reducing token-level baseline.
Scalable exploration
Exploration that matches Thompson sampling with only Θ(d log T) ensemble directions, and human-level Atari with 15% of DDQN’s interaction data.
Experience
- Dec 2025 – xAI (now SpaceX Super Intelligence) · Member of Technical Staff Stability, efficiency, optimizer and scaling laws; long-horizon agentic RL; on-policy distillation; RL infrastructure.
- 2025 ByteDance · LLM Algorithm Research Scientist Led RL training stability and efficiency for LLM reasoning and agents; co-led projects on multi-turn tool-integrated reasoning and GPU kernel generation.
- 2019 – 2022 Tencent AI & Robotics X · Student Research Staff Scalable exploration and stable off-policy deep RL.
- 2016 Microsoft Research, Theory Center · Research Intern
Selected publications
Get in touch
Open to conversations on research collaboration, advising, and speaking.
szrlee [at] gmail [dot] com