#063 - Prof. YOSHUA BENGIO - GFlowNets, Consciousness & Causality

We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST to get started creating a centralised, system of record for your team's machine learning work. Patreon:   / mlst   Discord:   / discord   For Yoshua Bengio, GFlowNets are the most exciting thing on the horizon of Machine Learning today. He believes they can solve previously intractable problems and hold the key to unlocking machine abstract reasoning itself. This discussion explores the promise of GFlowNets and the personal journey Prof. Bengio traveled to reach them. Pod version (with no music): https://anchor.fm/machinelearningstre... Our special thanks to: Alexander Mattick (Zickzack) References: Yoshua Bengio @ MILA (https://mila.quebec/en/person/bengio-...) GFlowNet Foundations (https://arxiv.org/pdf/2111.09266.pdf) Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation (https://arxiv.org/pdf/2106.04399.pdf) Interpolation Consistency Training for Semi-Supervised Learning (https://arxiv.org/pdf/1903.03825.pdf) Towards Causal Representation Learning (https://arxiv.org/pdf/2102.11107.pdf) Causal inference using invariant prediction: identification and confidence intervals (https://arxiv.org/pdf/1501.01332.pdf) A simple introduction to Markov Chain Monte–Carlo sampling https://link.springer.com/content/pdf... [00:00:00] Housekeeping [00:01:20] Weights and Biases sponsor clip [00:03:26] GFlowNets Introduction [00:16:24] Interview kick off [00:19:18] Galton Board Analogy [00:22:20] Free Energy Principle Connection [00:26:37] Diversity Preservation and Evolutionary Algorithms [00:28:25] The multi-armed bandit perspective [00:30:37] Avoiding Deception, Finding Unknown Unknows [00:33:53] Where GFlowNets Find Free Lunch [00:36:20] AlphaZero vs FlowZero (GFlowNets on Chess) [00:40:08] Using GFlowNets for Interactive Search [00:42:55] Learning Casaul Models as Graphs [00:46:39] Learning Abstract World Models [00:51:05] Can Machines Meta-Learn Categories [00:54:22] The Consciousness Prior. Is GPT-3 Conscious? [00:58:18] A Question For David Chalmers [01:01:25] Why are linear models dominating? They are abstraction! [01:05:23] Prof. Bengios Personal Journey (with Gary Marcus reference) [01:10:02] Debrief: A Dream Come True! [01:17:21] Abstraction is a Key [01:21:27] A Funny Definition of Causal [01:25:04] Arguing Semantics with a Semanticist [01:30:07] Human Learning Over Evolutionary Time Scales