SIPTA Seminar by Krikamol Muandet: Imprecise generalisation

ABSTRACT: The ability to generalise knowledge across diverse environments stands as a fundamental aspect of both biological and artificial intelligence (AI). In recent years, significant advancements have been made in out-of-domain (OOD) generalisation, including the development of new algorithmic tools, theoretical advancements, and the creation of large-scale benchmark datasets. However, unlike in-domain (IID) generalisation, OOD generalisation lacks a precise definition, leading to ambiguity in learning objectives. In this talk, I will explain how the tools from imprecise probability (IP) can be used to overcome the aforementioned ambiguity. Unlike the in-domain counterpart, the OOD generalisation is challenging because it involves not only learning from empirical data but also deciding among various notions of generalisation, i.e., worst-case, average-case, and interpolation thereof. Consequently, the learners face imprecision over the right notion of generalisation when there is an institutional separation between machine learners (e.g., ML engineers) and model operators (e.g., doctors), a common problem in practical applications of machine learning. To address these challenges, I will then introduce the concept of imprecise learning, drawing connections to imprecise probability, and discuss our initial work in the context of domain generalisation (DG), hypothesis testing, and truthful elicitation of imprecise forecasts. By exploring the synergy between learning algorithms and decision-making processes, this talk aims to shed light on the potential impact of IP in machine learning, paving the way for future advancements in the field. This talk is part of a series of seminars on imprecise probabilities that are organized by SIPTA, the "Society for Imprecise Probabilities: Theories and Applications". We also organize conferences and schools, provide documentation and maintain a mailing list and blog. More information is available at http://sipta.org. Info on the SIPTA seminars in particular is available at http://sipta.org/events/sipta-seminars Contents 00:00 - Start 02:37 - Introduction 09:09 - Empirical Risk Minimisation 23:23 - Domain Generalisation 41:49 - Domain Generalisation via Imprecise Learning 54:05- Precise vs Imprecise Learning 01:02:14 - Conclusion