Key Contributions

One notable project is “HyperAgent,” designed to quantify and resolve epistemic uncertainty on optimal value $Q^\star$ for scalable real-time sequential decision-making. HyperAgent demonstrates significant gains in data and computational efficiency in large-scale deep reinforcement learning (RL) benchmarks, such as the Atari suite. It has also shown effectiveness in human-AI alignment and collaboration, such as GPT-HyperAgent for content moderation with human feedback. Theoretical analysis of HyperAgent confirms that with logarithmic per-step computational complexity, its performance matches exact Thompson sampling (TS) in linear contextual bandits and Randomized Least-Square Value Iteration (RLSVI) in tabular RL environments. This analysis is grounded in the first probability tool for sequential random projection that I developed.

• HyperAgent: Efficient, scalable real-time decision-making.
  • Data and computation efficiency: Significant gains in deep RL benchmarks.
  • Applications: Human-AI alignment, e.g., content moderation with human feedback.
  • Theory: Matches TS and RLSVI with logarithmic computational complexity, proved via fundamental probability tools I developed.

Another key area of my research is game-theoretic decision-making, focusing on minimizing adversarial regret in repeated unknown games. This includes real-world applications where strategic agents learn to collaborate in traffic routing and compete in radar sensing. In these applications, the utility function is typically unknown at the start of the repeated games and requires learning from feedback after the strategic moves of multiple agents. Additionally, while the opponent’s strategic behavior is initially unknown, the revealed actions and game outcomes can be observed. Real-world applications usually exhibit special game structures due to domain prior knowledge, such as the structural properties of the utility function that depend on the joint actions of each agent and the opponents’ history-dependent strategic behavior. For example, the utility functions in traffic routing and radar sensing have polynomial and linear structures, respectively. I have developed frameworks that integrate structure-aware modeling with no-regret learning and optimization, resulting in significant sample budget savings.

• Game-theoretic Decision-making: Minimizing adversarial regret in repeated unknown games.
  • Applications: Collaboration in traffic routing and competition in radar sensing, achieving significant budget savings.
  • Frameworks: Synergy between domain knowledge-enhanced modeling and no-regret learning/optimization.

Currently, I am developing reliable and safe solutions for healthcare operations through inference-time algorithms. These algorithms leverage powerful cloud computing services on foundation models while augmenting necessary algorithmic modules in end devices. For example, controlling large language model (LLM) decoding towards high outcome feedback and minimum constraint violations via learned $Q^\star$. This is especially important for goal-conditioned sequential decision tasks that involves the LLMs while ensuring safety. Specifically, in healthcare inpatient flows, we employ LLM as multi-turn conversational agents to help doctor for various tasks and meanwhile enforces LLM agent to follow established rules.

• Reliable and Safe Operations: for healthcare, customer and business services.
  • Inference-time algorithms: Leveraging cloud services and augmenting end devices.
  • Control of LLM decoding: Guided by learned $Q^\star$ for high outcome feedback and minimum constraint violations.

This research aims to advance the field of interactive agents, contributing to both theoretical understanding and practical applications.