Regret comparable to exact Thompson sampling with only Θ(d log T) ensemble directions, kept by incremental updates.
Human-level Atari with 15% of DDQN's interaction data (1.5M), as a small addition to DQN.
A martingale extension of the Johnson–Lindenstrauss lemma to sequentially dependent, adaptively chosen data.
Traced LLM-RL collapse to the gap between the inference engine and the training engine.
A sequence-level trust region for long-horizon LLM-RL; a first-class algorithm in veRL.
The mismatch is not a static numerical error; it moves with the size of the update.
Safe reuse of off-policy data through a divergence between state distributions.
A per-token baseline weighted by realized gradient energy; group size 4 matches 32.
Sequence-level importance sampling and masking for general off-policy training.
A 14B kernel agent; ahead of Claude-4.5-Sonnet and GPT-5 on KernelBench Level 2 with test-time scaling.