Recent Advancements in Quantization, Pruning and Knowledge Distillation

Abstract:

In the rapidly evolving realm of Artificial Intelligence, where models are consistently growing in complexity, model compression has emerged as a strategic solution to address challenges arising from constrained resources and the need for efficient deployment. This presentation is dedicated to unveiling the pioneering efforts undertaken by Cisco Research in the field of model compression.

The key domains explored in this talk encompass:

• Introduction to Model Compression: Offering a comprehensive overview of the fundamental concepts that underpin model compression, elucidating the motivations behind reducing model size, computational demands, and memory consumption while upholding performance standards.
• Introducing Cisco’s Efficient AI Toolbox – ModelSmith: Presenting an introduction to Cisco’s innovative toolbox, ModelSmith, designed to facilitate efficient AI processes and model compression.
• Recent Advancements in Quantization, Pruning, and Knowledge Distillation from Cisco Research: Delving into Cisco Research’s recent strides in the domains of quantization, pruning, and knowledge distillation, shedding light on how these techniques enhance model efficiency and deployment.

Through this talk, the aim is to equip the audience with insights into Cisco’s contributions to the realm of model compression.  Whether you’re an AI researcher, developer, or enthusiast, this presentation offers a valuable perspective on how model compression is shaping the future of efficient AI deployment.

Bio:

Gaowen Liu is a researcher at Cisco research. She earned her Ph.D. in Computer Science at the University of Trento in 2017 and M.S. at the university of Trento and Nanjing University of Science and Technology. She was a visiting scholar at the Carnegie Mellon University and the University of Michigan. She has published 20+ research papers in the fields of computer vision, machine learning and multimedia. She received IBM best student Paper Award in ICPR 2014 and ICMR 2014 Student Travel Grant. Her main research interests relate to the investigation and implementation of new techniques in the fields of computer vision, multimedia and efficient AI. Specifically, She addresses a large spectrum of themes including model compression, human-behavior analysis, action recognition, object detection, etc. The specific research topics include cross media retrieval, multi-modal learning, social media analysis, cross-modal generation, etc.