Projects
Bayesian Inverse Games with High-Dimensional Multi-Modal Observations
Model Comparison
Video Loading Error
The video file may be using an unsupported codec. Please try:
- Re-encoding the video with H.264 codec
- Using a different browser
- Checking the video file format
This paper studies inverse games, where the goal is to infer latent agent objectives from observed behavior in strategic multi-agent settings. Rather than assuming perfect knowledge of agent intent, the framework explicitly models uncertainty and incorporates high-dimensional, multi-modal observations, combining trajectory data with visual scene information. By treating intent inference as a Bayesian inverse problem and differentiating through a dynamic game solver, the approach enables principled reasoning about ambiguity in complex autonomous driving scenarios.
Work
- Developed inverse dynamic game formulations for multi-agent systems, modeling agent interactions through generalized Nash equilibria
- Designed a Bayesian inference framework that represents latent agent intent probabilistically rather than deterministically
- Built a neural inference architecture combining a Variational Autoencoder (VAE) with DiNOv3-based visual embeddings to process high-dimensional image observations
- Integrated perception and dynamics by conditioning the inverse game on both trajectories and learned visual scene representations
- Implemented and evaluated the framework in CARLA, enabling realistic multi-agent autonomous driving experiments
- Performed Monte Carlo evaluations to quantify uncertainty, robustness, and inference accuracy across ambiguous interaction scenarios
Impact
This work demonstrates that incorporating uncertainty and multi-modal perception into inverse games significantly improves intent inference in interactive environments. The resulting predictions are more robust to ambiguity, making the approach well-suited for safety-critical autonomous systems where incorrect assumptions about agent intent can lead to failure.
Moncrief Summer Research Internship — Multi-Agent Inverse Games
Poster
This project explored how agent intent can be inferred from multi-modal observations—specifically top-down images and partial trajectory data—using simplified driving scenarios. The focus was on demonstrating, in a controlled toy environment, that combining visual context with motion information enables better reasoning about agent goals than trajectories alone.
Program
The Moncrief Summer Research Internship at the Oden Institute is a competitive undergraduate research program that immerses students in full-time computational science research under faculty mentorship, culminating in a formal poster presentation.
Project Summary
This project explored how agent intent can be inferred from multi-modal observations—specifically top-down images and partial trajectory data—using simplified driving scenarios. The focus was on demonstrating, in a controlled toy environment, that combining visual context with motion information enables better reasoning about agent goals than trajectories alone.
Work
- Proposed an inference framework that estimates agent intent from top-down image observations and trajectory data
- Formulated and solved an amortized maximum likelihood problem by differentiating through a Nash game solver
- Implemented sequence modeling with LSTMs to encode trajectory information
- Designed custom VAEs to represent latent intent variables
- Built CNN-based perception models to extract structured information from image observations
- Validated the approach in toy multi-agent driving environments to demonstrate improved intent recovery under ambiguity
Impact
This work provides an early demonstration that multi-modal perception materially improves inverse game inference, even in minimal settings. The results motivate extending inverse game frameworks beyond trajectory-only data and toward richer perceptual inputs, laying groundwork for safer and more informed multi-agent decision-making in autonomous systems.