Reinforcement Learning

An information-theoretic analysis showing that scalar advantage formulations learn ≤ log₂(B) bits per episode, while per-timestep advantages preserve full reward entropy.

The relentless push for faster inference creates a dangerous training-inference mismatch that silently kills RL with LLMs. We reveal the vicious cycle—particularly acute in reasoning and agentic RL—and show that sequence-level importance sampling is the principled solution.

Practically and provably efficient RL under resource constraints!

Practically and provably efficient RL under resource constraints!

Addressing efficiency chanllenge in RL by HyperFQI algorithm