AI Seminar
Web Agents: A New Frontier for Embodied Agents
This event is free and open to the publicAdd to Google Calendar
AI Seminar and Guest Lecture at CSE595 NLP Fall 2024
Location: George G. Brown Laboratories (GGBL) Building, Room 2505
Zoom: https://umich.zoom.us/j/95898538621
Meeting ID: 958 9853 8621
Passcode: aiseminar
Abstract:
The digital world, e.g., the World Wide Web, provides a powerful yet underexplored form of embodiment for AI agents, where perception involves understanding GUI renderings and the underlying markup languages, and effectors are mice and keyboards. Empowered by large language models (LLMs), web agents are rapidly rising as a new frontier for embodied agents that provide both the breadth and depth needed for driving agent development. On the other hand, web agents can potentially lead to many practical applications, thus also raising substantial commercial interest. In this talk, I will present an overview of web agents, covering the history, the promises, and the challenges. I will then give an in-depth discussion on multimodality and grounding. Finally, I will conclude with an outlook of promising future directions, including planning, synthetic data, and safety.
Bio:
Yu Su is a Distinguished Assistant Professor at the Ohio State University and a researcher at Microsoft. He founded and directs the OSU NLP group. He has broad interests in artificial and biological intelligence, with a primary interest in the role of language as a vehicle of thought and communication. His group is a leading force of LLM-based language agents, with many well-received works such as Mind2Web, SeeAct, HippoRAG, and MMMU. His work has received multiple paper awards, including Best Student Paper Award from CVPR and Outstanding Paper Award from ACL.