Yingru (Richard) Li
I make reinforcement learning on frontier models stable, efficient, and scalable.
I am a core contributor to RL fundamentals at xAI (now SpaceX Super Intelligence), contributing to the Grok RL big runs and to one of the longest stable RL runs to date. Before that I led RL stability and efficiency research for LLM reasoning and agents at ByteDance.
Ph.D. in Computer and Information Engineering, The Chinese University of Hong Kong (2025), advised by Prof. Zhi-Quan (Tom) Luo, with Prof. Benjamin Van Roy (Stanford & DeepMind) on my thesis committee.
Highlights
Why LLM-RL collapses, and the fix
One of the first (Sep 2025) to trace LLM-RL collapse to the training–inference mismatch, an off-policy bias. Our fix, masked importance sampling (MIS), made principled as Trust Region Masking (TRM), masks sequences without token clipping, so token scores stay centered. Adopted, shipped, or discussed by:
Open-source RL infrastructure
Authored the Rollout Correction framework for general off-policy training in veRL (20k+ GitHub stars), where MIS and TRM ship, and contributed the Optimal Token Baseline, its variance-reducing token-level baseline.
Data‑efficient exploration at scale
Exploration whose cost stays one SGD step as data grows: human-level Atari with 15% of DDQN’s interaction data, online language tasks on a GPT-2 backbone ahead of standard ensembles, and Thompson-sampling regret with only Θ(d log T) ensemble directions.
Experience
- Dec 2025 – xAI (now SpaceX Super Intelligence) · Member of Technical Staff Stability, efficiency, optimizer and scaling laws; long-horizon agentic RL; on-policy distillation; RL infrastructure.
- 2025 ByteDance · LLM Algorithm Research Scientist Led RL training stability and efficiency for LLM reasoning and agents; co-led projects on multi-turn tool-integrated reasoning and GPU kernel generation.
- 2019 – 2022 Tencent AI & Robotics X · Student Research Staff Data-efficient exploration and stable off-policy deep RL.
- 2016 Microsoft Research, Theory Center · Research Intern
Selected publications
Get in touch
Open to conversations on research collaboration, advising, and speaking.
szrlee [at] gmail [dot] com