SIPTA Seminar by Marnix Suilen: Robust Partially Observable Markov Decision Processes

ABSTRACT: Partially observable Markov decision processes (POMDPs) are the standard mathematical model for decision-making under uncertainty. POMDPs capture two forms of uncertainty: observations hide the true state from the decision-making agent, and probabilities make the outcome of each decision uncertain. Yet, standard POMDPs rely on the assumption that the probabilities with which observations and successor states of decisions are drawn are precisely known. Robust POMDPs (RPOMDPs) alleviate this assumption by making these probabilities imprecise. So-called uncertainty sets define a set of admissible probability distributions the model may operate under, and the decision-making agent’s goal is to optimize its policy under the worst-case instance of this uncertainty set. In this talk, I will give an introduction to RPOMDPs, with a particular focus on their semantics and algorithms. This talk is part of a series of seminars on imprecise probabilities that are organized by SIPTA, the "Society for Imprecise Probabilities: Theories and Applications". We also organize conferences and schools, provide documentation and maintain a mailing list and blog. More information is available at http://sipta.org. Info on the SIPTA seminars in particular is available at http://sipta.org/events/sipta-seminars Contents 00:00 - Start 01:48 - Introduction 09:23 - Partially observable markov decision processes 14:37 - Robust POMDP 25:35- Robust POMDP semantics 33:51 - Algorithms for robust POMDP 45:10 - Conclusions