Jason Corso receives Google Faculty Research Award

Prof. Jason Corso received a Google Faculty Research Award to further his research in computational learning from instructional video content. His goal is to develop a consistent and reliable method for producing a visual and textual summary of any video that describes a process – from simple sandwich how-to’s to more elaborate technical processes.

Prof. Corso believes that this research could make it easier to search for certain types of videos on the web, especially on a site such as YouTube. In addition, he says his research could enable artificial intelligence agents, such as service robots, to learn typical human processes as naturally as humans do by leveraging terabytes of this rich instructional video content.

He plans to build his expertise using instructional cooking videos, starting with the deceptively simple problem of making a peanut butter and jelly sandwich. There is very little existing work on machine learning from video. Some of the peskier problems include: how to allow for variations in a single process (e.g., does it matter whether the peanut butter or jelly gets spread first); what irrelevant information in a video should be disregarded (e.g., do we care whether it is grape or strawberry jelly); and how to detect certain partially visible objects, such as the knife needed to spread the PB&J.

The results of the investigation will further his existing research on the topic, which he has already made available to other researchers via his website. This includes his work on learning from instructional cooking videos and his YouCook dataset. YouCook is an annotated data set of unconstrained third-person cooking videos that was prepared from 88 open-source YouTube cooking videos.

Prof. Corso is an Associate Professor of Electrical and Computer Engineering. He conducts research in computer vision, medical imaging, data mining, machine learning, artificial intelligence and ontology. He is a recipient of the NSF CAREER award (2009), ARO Young Investigator award (2010), and is a member of the DARPA Computer Science Study Group, which supports young faculty to facilitate development of transformational technologies in computer science and related fields.

About the Google Faculty Research Award

The Google Faculty Research Awards program is a competitive worldwide program intended to facilitate more interaction between Google and academia. The intent of the awards program is to support academic research that is aligned with Google’s mission. The awards are one-year awards structured as unrestricted gifts to universities to support the work of world-class full-time faculty members at top universities around the world.