Human-Agent Interaction and Simulation
Simulated users, AI patients, speech interaction, financial agents, and micro-world environments for studying how agents behave with people and worlds.
Human-agent interaction is the connective tissue between model capability and real deployment. This project line studies agents that interact with people, learners, patients, investors, speech partners, and simulated worlds, where behavior unfolds over time rather than in a single static prompt.
Background and Motivation
Many failures only appear after several turns: users change goals, agents adapt, environments respond, and earlier decisions constrain later ones.
Medical learners, investors, patients, and speech users bring beliefs, emotions, incentives, and incomplete information into the loop.
Before deploying agents in clinics, markets, classrooms, or social settings, we can study them in controlled and instrumented environments.
Research Storyline
LLM user simulators and Socratic questioning systems generate interactive training environments for multi-turn dialogue.
AI standardized patients support repeatable practice in history taking, communication, and clinical reasoning.
TwinMarket and Economic World Models use agent societies to examine emergent markets, policy sandboxes, and collective risk.
MicroVerse extends interaction to scientific worlds, where agents must reason about hidden mechanisms and changing biological states.
Project Clusters
Large Language Model as a User Simulator and PlatoLM study how simulated users can teach, probe, and improve dialogue models.
EasyMED and AI standardized patients connect human-agent interaction with medical training and learner-centered evaluation.
TwinMarket and Economic World Models make multi-agent interaction observable at social, financial, and policy scale.
S2S-Arena, EchoMind, and speech-to-speech Turing tests examine whether agents can interact naturally through voice and paralinguistic signals.
Typical Work
Uses LLMs to simulate users and generate interactive training or evaluation data for dialogue systems.
PaperStudies Socratic questioning as a way to teach multi-turn dialogue behavior through simulated interaction.
PaperBuilds a scalable behavioral and social simulation for financial markets with LLM-based investor agents.
Project pageExplores micro-world simulation where visible phenomena depend on hidden scientific mechanisms and evolving states.
Project pageDisplay Figures
Resource Map
Umbrella project for agentic economies, policy sandboxes, and self-correcting economic twins.
Project pageAI standardized patient project for repeatable and benchmarked medical learning scenarios.
Project pageVoice, paralinguistic, empathy, and embodied interaction benchmarks and datasets.
Project page