AI Seminar

Web Agents: A New Frontier for Embodied Agents

Yu SuAssistant Professor of Engineering Inclusive Excellence at the Department of Computer Science and EngineeringOhio State University
WHERE:
George G. Brown Laboratories (GGBL) Building, Room 2505
SHARE:

AI Seminar and Guest Lecture at CSE595 NLP Fall 2024

Location: George G. Brown Laboratories (GGBL) Building, Room 2505

Zoom: https://umich.zoom.us/j/95898538621

Meeting ID: 958 9853 8621

Passcode: aiseminar

 

Abstract:

The digital world, e.g., the World Wide Web, provides a powerful yet underexplored form of embodiment for AI agents, where perception involves understanding GUI renderings and the underlying markup languages, and effectors are mice and keyboards. Empowered by large language models (LLMs), web agents are rapidly rising as a new frontier for embodied agents that provide both the breadth and depth needed for driving agent development. On the other hand, web agents can potentially lead to many practical applications, thus also raising substantial commercial interest. In this talk, I will present an overview of web agents, covering the history, the promises, and the challenges. I will then give an in-depth discussion on multimodality and grounding. Finally, I will conclude with an outlook of promising future directions, including planning, synthetic data, and safety.

Bio:

Yu Su is a Distinguished Assistant Professor at the Ohio State University and a researcher at Microsoft. He founded and directs the OSU NLP group. He has broad interests in artificial and biological intelligence, with a primary interest in the role of language as a vehicle of thought and communication. His group is a leading force of LLM-based language agents, with many well-received works such as Mind2Web, SeeAct, HippoRAG, and MMMU. His work has received multiple paper awards, including Best Student Paper Award from CVPR and Outstanding Paper Award from ACL.

Organizer

AI Lab

Student Host

Martin Ziqiao MaAI Lab Seminar Tsar

Faculty Host

Joyce ChaiProfessor, Computer Science and EngineeringUniversity of Michigan