Yingru (Richard) Li
Learning from experience, at frontier scale.
Core contributor to RL fundamentals at xAI (now SpaceX Super Intelligence): the science of RL training behind the Grok RL big runs, including one of the longest stable RL runs to date. Previously led RL stability and efficiency research at ByteDance.
PhD, The Chinese University of Hong Kong (2025), advised by Prof. Zhi-Quan (Tom) Luo, with Prof. Benjamin Van Roy (Stanford & DeepMind) on my thesis committee.
Highlights
Why LLM-RL collapses, and the fix
Among the first to trace LLM-RL collapse to the training–inference mismatch.
First published in September 2025. Our fix, masked importance sampling (MIS), made principled as Trust Region Masking (TRM), is adopted, shipped or discussed by:
The science of RL training at frontier scale
Optimization, scaling laws, stability and efficiency of RL on frontier models.
Core algorithms and training recipes, optimizer behavior and numerics at scale, and first-principles theory of how an update moves a model and how training should scale.
Open-source RL infrastructure
Authored veRL’s Rollout Correction framework and its Optimal Token Baseline.
veRL has 20k+ GitHub stars. Rollout Correction is its framework for off-policy training, and MIS and TRM ship in it.
Agentic RL for GPU kernel generation
Dr. Kernel-14B writes Triton kernels and refines them using execution feedback.
Its multi-turn RL pairs an unbiased turn-level advantage with our mismatch correction against early collapse. With test-time scaling, 31.6% of its KernelBench Level 2 kernels run at least 1.2× faster than PyTorch, ahead of Claude-4.5-Sonnet (26.7%) and GPT-5 (28.6%).
Data‑efficient exploration at scale
Exploration whose cost stays one SGD step as data grows.
Human-level Atari with 15% of DDQN’s interaction data, and Thompson-sampling regret with only Θ(d log T) ensemble directions.
Experience
- Dec 2025 – xAI (now SpaceX Super Intelligence) · Member of Technical Staff RL fundamentals for the Grok RL big runs; long-horizon agentic RL; on-policy distillation; RL infrastructure.
- 2025 ByteDance · LLM Algorithm Research Scientist Led RL training stability and efficiency for LLM reasoning and agents; co-led projects on multi-turn tool-integrated reasoning and GPU kernel generation.
- 2019 – 2022 Tencent AI & Robotics X · Student Research Staff Data-efficient exploration and stable off-policy deep RL.
- 2016 Microsoft Research, Theory Center · Research Intern
Selected publications
Get in touch
Open to conversations on research collaboration, advising, and speaking.