The reward hypothesis | Richard Sutton & Julia Haas | Absolutely Interdisciplinary 2023

Almost 20 years ago, AI research pioneer Richard Sutton posited the reward hypothesis: “That all of what we mean by goals and purposes can be well thought of as maximization of the expected value of the cumulative sum of a received scalar signal (reward).” Since then, advances in reinforcement learning have demonstrated that complex behaviours can emerge from artificial agents guided by scalar reward. Humanists and social scientists are starting to see the utility of the hypothesis as a claim about humans, although many disagree. The question remains, is the reward hypothesis of reinforcement learning a good model for understanding human behaviour and values? How far can it go? Can it guide normative decision-making for individuals and groups? For societies? Speakers: Julia Haas, Gillian Hadfield (moderator), Richard Sutton Julia Haas is a senior research scientist in the Ethics Research Team at DeepMind. Haas was previously an assistant professor in the Department of Philosophy and the Neuroscience Program at Rhodes College and an affiliated researcher with ANU's Humanizing Machine Intelligence Grand Challenge. She was also a research fellow in the School of Philosophy at the Australian National University and a McDonnell Postdoctoral Research Fellow in the Philosophy Neuroscience Psychology program at Washington University in St. Louis. Haas’s research is in the philosophy of cognitive science and neuroscience. She works on the nature of valuation and its role in theories of the mind. Her current work includes investigating the possibility of meaningfully moral artificial intelligence. Richard S. Sutton is one of the pioneers of reinforcement learning, a field in which he continues to lead the world. He is most interested in understanding what it means to be intelligent, to predict and influence the world, to learn, perceive, act, and think. He seeks to identify general computational principles underlying what we mean by intelligence and goal-directed behaviour. Sutton currently seeks to extend reinforcement learning ideas to an empirically grounded approach to knowledge representation based on prediction. Sutton is chief scientific advisor, a fellow and Canada CIFAR AI Chair at Amii, a professor of computing science at the University of Alberta, and a distinguished research scientist at DeepMind. Sutton has been named a Fellow of the Royal Society of Canada, the Association for the Advancement of Artificial Intelligence (AAAI), and the Canadian Artificial Intelligence Association (CAIAC), where he also received a Lifetime Achievement Award in 2018. About Absolutely Interdisciplinary: Absolutely Interdisciplinary sets out to generate new conversations and insights by pairing scholars from different disciplines to address a common research question. By examining the impacts of technical systems on society through multiple perspectives, the conference aims to foster interdisciplinary dialogues to better understand how AI can promote human well-being for everyone. Learn more: https://absolutelyinterdisciplinary.c... 0:00 Intro 3:32 Richard Sutton, "Reward and Related Reductionist Hypotheses" 44:27 Julia Haas, "Reward, Value, & Minds Like Ours" 1:13:16 Discussion 1:31:16 Q&A