Yingru (Richard) Li
Learning from experience, at frontier scale.
Core contributor to RL fundamentals at xAI (now SpaceX Super Intelligence): the science of RL training behind the Grok RL big runs, including one of the longest stable RL runs to date. Previously led RL stability and efficiency research at ByteDance.
PhD, The Chinese University of Hong Kong (2025), advised by Prof. Zhi-Quan (Tom) Luo, with Prof. Benjamin Van Roy (Stanford & DeepMind) on my thesis committee.
Highlights
Why LLM-RL collapses, and the fix
Among the first to trace LLM-RL collapse to the training–inference mismatch.
First published in September 2025. Our fix, masked importance sampling (MIS), made principled as Trust Region Masking (TRM), is adopted, shipped or discussed by:
The science of RL training at frontier scale
Optimization, scaling laws, stability and efficiency of RL on frontier models.
Core algorithms and training recipes, optimizer behavior and numerics at scale, and first-principles theory of how an update moves a model and how training should scale.
Open-source RL infrastructure
Authored Rollout Correction and Optimal Token Baseline in veRL (20k+ GitHub stars).
Rollout Correction is veRL’s framework for off-policy training, and MIS and TRM ship in it.
Agentic RL for GPU kernel generation
A 14B Triton-kernel agent that, with test-time scaling, beats GPT-5 on KernelBench Level 2.
Dr. Kernel-14B writes Triton kernels and refines them using execution feedback. Its multi-turn RL pairs an unbiased turn-level advantage with our mismatch correction against early collapse. With test-time scaling, 31.6% of its KernelBench Level 2 kernels run at least 1.2× faster than PyTorch, ahead of Claude-4.5-Sonnet (26.7%) and GPT-5 (28.6%).
Data‑efficient exploration at scale
Exploration whose cost stays one SGD step as data grows.
Human-level Atari with 15% of DDQN’s interaction data, and Thompson-sampling regret with only Θ(d log T) ensemble directions.
Experience
- Dec 2025 – xAI (now SpaceX Super Intelligence) · Member of Technical Staff RL fundamentals for the Grok RL big runs; long-horizon agentic RL; on-policy distillation; RL infrastructure.
- 2025 ByteDance · LLM Algorithm Research Scientist Led RL training stability and efficiency for LLM reasoning and agents; co-led projects on multi-turn tool-integrated reasoning and GPU kernel generation.
- 2019 – 2022 Tencent AI & Robotics X · Student Research Staff Data-efficient exploration and stable off-policy deep RL.
- 2016 Microsoft Research, Theory Center · Research Intern
Selected publications
Things I believe in
- Learning from experience is the part of intelligence that keeps going.
- Stable runs are designed. Collapses have causes.
- Write the derivation down; the bug is usually in the step you skipped.
- The smallest example that shows the failure is the best place to start.
- Uncertainty is worth acting on.
I started RL-Seminar, an RL theory seminar, in 2019 and ran it through 2023.
Get in touch
Open to conversations on research collaboration, advising, and speaking. I read every email and write back.