Magentic-UI is an open-source research prototype developed by Microsoft Research, designed to facilitate human-centered interaction with AI agents for web-based tasks. Unlike fully autonomous agents, Magentic-UI emphasizes transparency, control, and collaboration, enabling users to actively participate in task planning and execution. Built on AutoGen and integrated with Azure AI Foundry Labs, it offers advanced capabilities for navigating the web, executing code, and processing files, while prioritizing safety and user oversight.
The core features of Magentic-UI include co-planning, where users can collaboratively refine step-by-step task plans before execution; co-tasking, allowing users to intervene during task execution by providing feedback or taking direct control; action guards, which ensure user approval for potentially irreversible actions; and plan learning, enabling the system to save and reuse successful task plans for future scenarios. These functionalities work together to create a dynamic and interactive experience, where human input enhances the performance and reliability of the agent.
Magentic-UI operates through a modular architecture powered by specialized agents, including the Orchestrator for planning and delegation, WebSurfer for browser-based tasks, Coder for code execution, and FileSurfer for file processing. These agents collaborate under the supervision of the Orchestrator, which integrates user feedback and dynamically adjusts task plans as needed. All interactions are conducted within a sandboxed environment to ensure security, with features like Docker containers and configurable allow-lists preventing unauthorized actions or access.
The system’s design reflects a commitment to safety, transparency, and user control. Users can interrupt tasks at any time, configure action-approval policies, and set restrictions on web access. Additionally, Magentic-UI has undergone rigorous evaluations, including simulated user experiments and red-team testing, to assess its performance and resilience against malicious scenarios. Preliminary results demonstrate significant improvements in task accuracy when human feedback is incorporated, bridging the gap between autonomous agent performance and human-level task completion.
Magentic-UI also serves as a research platform for exploring critical questions in human-agent collaboration, such as optimizing intervention efficiency and safeguarding against web-based attacks. By enabling researchers to study these challenges in a realistic setting, Magentic-UI aims to advance the field of agentic systems while fostering innovation in human-centered AI design.
