Yingru (Richard) Li
Learning from experience, at frontier scale.
Core contributor to RL fundamentals at xAI (now SpaceX Super Intelligence), on the Grok RL big runs and one of the longest stable RL runs to date. Previously led RL stability and efficiency research at ByteDance.
PhD, The Chinese University of Hong Kong (2025), advised by Prof. Zhi-Quan (Tom) Luo, with Prof. Benjamin Van Roy (Stanford & DeepMind) on my thesis committee.
Highlights
Why LLM-RL collapses, and the fix
Among the first (Sep 2025) to trace LLM-RL collapse to the training–inference mismatch. Our fix, masked importance sampling (MIS), made principled as Trust Region Masking (TRM), is adopted, shipped or discussed by:
Open-source RL infrastructure
Authored two parts of veRL (20k+ GitHub stars): the Rollout Correction framework for off-policy training, where MIS and TRM ship, and the Optimal Token Baseline.
Data‑efficient exploration at scale
Exploration whose cost stays one SGD step as data grows: human-level Atari with 15% of DDQN’s interaction data, and Thompson-sampling regret with only Θ(d log T) ensemble directions.
Experience
- Dec 2025 – xAI (now SpaceX Super Intelligence) · Member of Technical Staff Stability, efficiency, optimizer and scaling laws; long-horizon agentic RL; on-policy distillation; RL infrastructure.
- 2025 ByteDance · LLM Algorithm Research Scientist Led RL training stability and efficiency for LLM reasoning and agents; co-led projects on multi-turn tool-integrated reasoning and GPU kernel generation.
- 2019 – 2022 Tencent AI & Robotics X · Student Research Staff Data-efficient exploration and stable off-policy deep RL.
- 2016 Microsoft Research, Theory Center · Research Intern
Selected publications
Get in touch
Open to conversations on research collaboration, advising, and speaking.