CLIMB Evergreen talk with Emmanuel Candès: Conformal Inference when Data is not Exchangeable

Title: Conformal Inference When Data is Not Exchangeable Speaker: Emmanuel J. Candès, Stanford University Abstract: Conformal inference methods are becoming all the rage in academia and industry alike. In a nutshell, these methods deliver exact prediction intervals for future observations without making any distributional assumption whatsoever other than having iid, and more generally, exchangeable data. However, even this minimal assumption may fail since in most settings the distribution of observations can shift drastically—think of finance or economics where market behavior can change in response to new legislation or major world events, or public health where changes occur because of policies. What are we to do then? This talk will introduce two new methods to deal with this situation. The first introduces a new method, which allows for regression algorithms that are not symmetric in the training data with no resulting loss of coverage if the data points are indeed iid/exchangeable. Next, if in fact the data points are not iid/exchangeable, then placing weights on the training data points before running the method enables the final prediction interval to be robust vias a vis changes in the distribution of the data. The second method had a flavor of an online algorithm: it operates by correcting naive prediction sets constructed by naive methods and produces prediction intervals that are robust to deviations occurring over time. We will illustrate our new methods via examples predicting election results or COVID19-case trajectories. This is joint work with Isaac Gibbs, and with Rina Barber, Aaditya Ramdas and Ryan Tibshirani. Bio: : Emmanuel Candès is the Barnum-Simons Chair in Mathematics and Statistics at Stanford University, and Professor of Electrical Engineering (by courtesy). His research interests lie at the interface of statistics, information theory, signal processing and computational mathematics. He received his Ph.D. in statistics from Stanford in 1998. Candès has received several awards including the Alan T. Waterman Award from NSF, which is the highest honor bestowed by NSF to early-career scientists, and the MacArthur Fellowship, popularly known as the ‘genius award’. He has given over 90 plenary lectures at major international conferences, not only in mathematics and statistics but in many other areas as well including biomedical imaging and solid-state physics. He was elected to the National Academy of Sciences and to the American Academy of Arts and Sciences in 2014. In 2020, he received the Princess of Asturias Award for Technical and Scientific Research. Event sponsored by CLIMB Center with support from FODSI (https://fodsi.us/) CLIMB The Center for the Theoretical Foundations of Learning, Inference, Information, Intelligence, Mathematics and Microeconomics at Berkeley (CLIMB) was established to address new conceptual and mathematical challenges arising at the interface between technology, science, and society. CLIMB recognizes the emergence of a new generation of technology that focuses on data, inferences, and decisions, a development that is leading to a deeper engagement between technology and real-world phenomena that involve human decisions, values, discoveries, and culture. Website: www.climb.berkeley.edu