Towards Embodied Visual Intelligence

Joint CSE/ECE Faculty Candidate Seminar

What would it mean for a machine to see the world?
Computer vision has recently made great progress on problems
such as finding categories of objects and scenes, and poses of
people in images. However, studying such tasks in isolated
disembodied contexts, divorced from the physical source of their
images, is insufficient to build intelligent visual agents. My
research focuses on remarrying vision to action, by asking: how
might vision benefit from the ability to act in the world, and vice
versa? Could embodied visual agents teach themselves through
interaction and experimentation? Are there actions they might
perform to improve their visual perception? Could they exploit
vision to perform complex control tasks? In my talk, I will set up
the context for these questions, and cover some strands of my
work addressing them, proposing approaches for self-supervised
learning through proprioception, visual prediction for
decomposing complex control tasks, and active perception.
Finally, I will discuss my long-term vision and directions that I
hope to work on in the next several years.
Dinesh Jayaraman is a postdoctoral scholar in EECS at
UC Berkeley. He received his PhD from UT Austin (2017) and B.
Tech from IIT Madras (2011). His research interests are broadly
in computer vision, robotics, and machine learning. In the last
few years, he has worked on visual prediction, active perception
and visual learning in embodied agents, visuo-tactile robotic
manipulation, semantic visual attributes, and zero-shot
categorization. His work has been recognized with the ACCV
Best Application Paper Award (2016), a Samsung PhD
Fellowship (2016), a UT Austin Graduate Dean's Prestigious
Fellowship (2016), and a UT Austin Microelectronics and
Computer Development Fellowship Award (2011). He reviews
for top conferences and journals across computer vision,
machine learning, and robotics, won a CVPR Outstanding
Reviewer Award (2016), and served as an Area Chair for NIPS
2018.