Friends Don’t Let Friends Deploy Black-Box Models: The Importance of Intelligibility in Machine Learning for HealthCare

Abstract:

In machine learning often a tradeoff must be made between accuracy and intelligibility: the most accurate models usually are not very intelligible, and the most intelligible models usually are less accurate. This often limits the accuracy of models that can safely be deployed in mission-critical applications such as healthcare where being able to understand, validate, edit, and ultimately trust a model is important. We have developed a learning method based on generalized additive models that is as accurate as full complexity models, but even more intelligible than linear models. This makes it easy to understand what a model has learned and to edit the model when it learns inappropriate things. Making it possible for medical experts to understand and repair a model is critical because most clinical data has unanticipated problems. In the talk I’ll present two healthcare case studies where these high-accuracy GAMs discover surprising patterns in the data that would have made deploying a black-box model risky. I’ll also show how we’re using these models to detect bias in domains where fairness and transparency are paramount, and how these models can be used to understand what is learned by black-box models such as deep nets.

Bio:

Rich Caruana is a Senior Principal Researcher at Microsoft.  His research focus is on intelligible/transparent modeling, machine learning for medical decision making, deep learning, and computational ecology.  Before joining Microsoft, Rich was on the faculty in Computer Science at Cornell, at UCLA’s Medical School, and at CMU’s Center for Learning and Discovery.  Rich’s Ph.D. is from CMU, where he worked with Tom Mitchell and Herb Simon.  His thesis on Multitask Learning helped create interest in a subfield of machine learning called Transfer Learning.  Rich received an NSF CAREER Award in 2004 (for Meta Clustering), best paper awards in 2005 (with Alex Niculescu-Mizil), 2007 (with Daria Sorokina), and 2014 (with Todd Kulesza, Saleema Amershi, Danyel Fisher, and Denis Charles), and co-chaired KDD in 2007 with Xindong Wu.