Gradient Descent vs Evolution | How Neural Networks Learn
Explore two learning algorithms for neural networks: stochastic gradient descent and an evolutionary algorithm known as a local search. They fundamentally solve the same problem in similar ways, but one has the advantage. Step-by-step they find a way down Loss Mountain. Watch real neural networks maximize the fitness of curve fitting. We've got Dogson here! Special thanks to Andrew Carr(https://x.com/andrew_n_carr) and Josh Greaves for reviewing this with their human neurons, and to the artificial neurons of Grok, o3 mini, and claude. Grok thought the gay joke was funny, o3 thought it wasn't inclusive lol. it is inclusive! ~Webtoys~ Hill Climbers: https://neuralpatterns.io/hill_climbe... Neuron Tuner: https://neuralpatterns.io/nn_tuner.html Subscribe to my music guy NOW: / @acolyte-compositions ~Links~ Patreon: / emergentgarden Kofi: https://ko-fi.com/emergentgarden My Twitter: / max_romana My Bluesky: https://bsky.app/profile/emergentgard... My Other NN videos: • Neural network Learns Webtoy Source: https://github.com/MaxRobinsonTheGrea... Animation Source: https://github.com/MaxRobinsonTheGrea... Image Approximators: https://github.com/MaxRobinsonTheGrea... FUNCTIONS DESCRIBE THE WORLD: • On Mathematical Maturity (1) Thomas Garrity Dawkins Climbing Mount Improbable: • Richard Dawkins demonstrates the evolution... But he's gay: • Gay Mount Everest ~Citations~ Unfortunately many of these are behind paywalls NNs are Universal Function Approximators: https://www.cs.cmu.edu/~epxing/Class/... Backpropagation: https://www.nature.com/articles/323533a0 Loss Surfaces of MLPs: https://arxiv.org/abs/1412.0233 ~Timestamps~ (0:00) Learning Learning (1:20) Neural Network Space (3:40) The Loss Landscape (7:21) The Blind Mountain Climber (8:37) Evolution (Local Search) (13:07) Gradient Descent (18:40) The Gradient Advantage (20:48) The Evolutionary (dis)advantage
All Machine Learning algorithms explained in 17 min
How I created an evolving neural network ecosystem
Watching Neural Networks Learn
![The Misconception that Almost Stopped AI [How Models Learn Part 1]](https://i.ytimg.com/vi/NrO20Jb-hy0/hqdefault.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLCiksXndIEYQZVVoTfArQwhou-eWw)
The Misconception that Almost Stopped AI [How Models Learn Part 1]
The Brain’s Learning Algorithm Isn’t Backpropagation
Creatures in Higher Dimensions
Why Neural Networks can learn (almost) anything
The Brain Eating Machines
Evolution in Higher Dimensions
The most complex model we actually understand
Neuroevolution Explained by Example
Pushing Simulations to the LIMIT to Find Order in Chaos
Transformers, the tech behind LLMs | Deep Learning Chapter 5
But what is a neural network? | Deep learning chapter 1
How Maxwell's Equations Were Discovered
The Tiny Donut That Proved We Still Don't Understand Magnetism
The Most Important Algorithm in Machine Learning
Emergent Complexity
Gradient descent, how neural networks learn | Deep Learning Chapter 2
