Why LeCun Thinks Deep Learning Isn't Enough — Yann LeCun

We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST Are neural networks truly learning smooth, continuous data manifolds, or are they merely performing a high-resolution, piecewise linear slice-and-dice of the input space? In this episode, we unpack the mathematical reality of deep learning with Prof. Yann LeCun and Dr. Randall Balestriero from Meta AI. We initially approached Balestriero and LeCun's paper, Learning in High Dimensions Always Amounts to Extrapolation, with heavy skepticism. However, this conversation fundamentally shifted our perspective on how neural networks operate. We explore why the traditional intuition of interpolation completely breaks down in high-dimensional spaces, meaning the classic convex hull definition of extrapolation is effectively broken when analyzing modern deep learning. Prof. Yann LeCun is the Chief AI Scientist at Meta, a Turing Award laureate, and widely recognized as a foundational figure in deep learning. Dr. Randall Balestriero is a researcher at Meta AI whose recent work on the spline theory of deep neural networks provides a rigorous geometrical framework for understanding their underlying mechanics. Main insights and topics discussed: The spline theory of deep learning and how networks with ReLU activations recursively partition the ambient space into polyhedral convex cells. Why neural networks are mathematically equivalent to compositions of linear functions, functioning more like locally sensitive hash tables than smooth geometric morphers. The illusion of the homogeneous latent space, revealing that neural networks actually create input-specific affine transformations. The curse of dimensionality and the mathematical impossibility of statistical generalization without severe inductive biases. How the machine learning community's reliance on piecewise linear functions challenges the assumption that networks discover complex, smooth nonlinear realities. --- TIMESTAMPS: 00:00:00 The Interpolation vs Extrapolation Debate 00:10:00 Weights and Biases Sponsorship 00:15:00 Spline Theory & Polyhedra in Neural Nets 00:25:00 The Piecewise Linear Reality of Deep Learning 00:35:00 Interpolative Representations & Feature Engineering 00:45:00 Convex Hulls and Local Generalization 00:55:00 The Curse of Dimensionality Visualized 01:05:00 Introducing Yann LeCun 01:10:00 LeCun on Why the Interpolation Dichotomy is Flawed 01:20:00 LeCun on Reasoning & Self-Supervised Learning 01:30:00 LeCun on Definitions of Interpolation & Attention 01:40:00 LeCun on Joint Embedding Architectures 01:50:00 LeCun on Energy Minimization & Model Predictive Control 02:01:00 Introducing Randall Balestriero & Redefining Interpolation 02:15:00 Balestriero on Why Deep Learning Actually Works 02:25:00 Balestriero on The Manifold Hypothesis & Separability 02:35:00 Balestriero on Spline Theory & Neural Decision Trees 02:45:00 Balestriero on MNIST vs ImageNet Dimensionality 02:55:00 Post-Interview Debrief: Interpolation & Reasoning 03:10:00 Post-Interview Debrief: Latent Space & The Lottery Ticket --- REFERENCES: Paper: [00:03:10] Learning in High Dimensions Always Amounts to Extrapolation https://arxiv.org/abs/2110.09485 [00:16:30] A Spline Theory of Deep Learning https://arxiv.org/abs/1802.06975 [00:50:40] Interpolation of Sparse High-Dimensional Data https://arxiv.org/abs/2006.13915 [01:42:10] BYOL (Bootstrap Your Own Latent) https://arxiv.org/abs/2006.07733 [01:43:10] Barlow Twins https://arxiv.org/abs/2103.03230 [01:44:10] VICReg https://arxiv.org/abs/2105.04906 [02:35:10] Neural Decision Trees https://arxiv.org/abs/1912.10098 Company: [00:10:10] Weights & Biases https://wandb.ai/ Website: [00:25:30] TensorFlow Playground https://playground.tensorflow.org/ Book: [00:35:20] Deep Learning with Python (Francois Chollet) https://www.amazon.com/Deep-Learning-... [01:53:20] Thinking, Fast and Slow (Daniel Kahneman) https://www.amazon.com/Thinking-Fast-... Person: [01:05:10] Yann LeCun https://yann.lecun.com/ [01:21:30] Geoffrey Hinton https://www.cs.toronto.edu/~hinton/ --- LINKS: Full Transcript: https://app.rescript.info/share/98ksv... Download PDF transcript: https://app.rescript.info/api/public/... Patreon:   / mlst   Discord:   / discord   Interpolation of Sparse High-Dimensional Data [Dr. Thomas Lux] https://tchlux.github.io/papers/tchlu...