Mobile Agent
Benchmarks, environments, safety tests, and training recipes for phone-use agents operating real mobile workflows.
Mobile Agent builds the infrastructure, benchmarks, and training recipes for phone-use agents that can operate mobile workflows reliably, privately, and safely. The research line moves beyond screenshot QA toward executable tasks, auditable traces, mixed action spaces, simulated phone environments, and real-app reinforcement learning.
Background and Motivation
Daily work already happens across mobile apps, permissions, notifications, files, accounts, and dynamic screens, making phones a realistic testbed for general-purpose agents.
Real workflows often require a mixture of GUI interaction, command-line execution, tool calls, state inspection, and verification beyond simple click trajectories.
An agent that completes a task while leaking data, overusing permissions, or taking unsafe actions is not deployment-ready.
Research Storyline
MyPhoneBench reframes phone-agent readiness around permissioned access, minimal disclosure, and user-controlled memory.
PhoneSafety evaluates safety-critical moments and distinguishes safe actions, unsafe actions, and failures to do anything useful.
PhoneWorld turns real GUI trajectories and screenshots into reusable environments with executable tasks, state transitions, and verifiers.
PhoneHarness gives phone agents a mixed-action execution framework with auditable traces and task-level verification.
PhoneBuddy combines real-app environments and mock environments so open models can learn robust agentic phone-use behavior.
Representative Work
Introduces MyPhoneBench for evaluating privacy risks in phone-use agents across realistic mobile tasks.
PaperRethinks safety evaluation by separating genuinely safe choices from unhelpful or failed actions.
PaperBuilds scalable phone-use environments from real GUI traces, controllable screens, executable tasks, and verifiers.
PaperExpands agent execution beyond GUI-only actions and evaluates whether mobile workflows are truly completed.
PaperUses real-app and mock phone environments to train open models for stronger agentic phone-use performance.
PaperTechnical Layers
Reusable mobile environments make real app workflows controllable, replayable, and measurable.
Agents can combine GUI operations, CLI commands, and tool calls while leaving traces that humans and verifiers can audit.
Benchmarks judge whether the external phone state changed correctly, not just whether the model produced plausible intermediate actions.
Evaluation tracks permission use, data disclosure, memory control, unsafe operations, and refusal behavior in mobile settings.
Display Figures
Resource Map
Project site for mixed-action phone-use agent harnessing and benchmark resources.
Project siteOpen-source implementation for the phone-use harness and evaluation workflow.
RepositoryDataset release for evaluating phone agents through mixed GUI, CLI, and tool-action traces.
Dataset