Shannon Entropy and Information Gain
CORRECTION: at 13:41, the probability is 6.1e-5 and not 4.8e-4 (however, the entropy is 1.75, which is correct). Thank you @dlyChimi! Learn Shannon entropy and information gain by playing a game consisting in picking colored balls from buckets. Announcement: New Book by Luis Serrano! Grokking Machine Learning. bit.ly/grokkingML 40% discount code: serranoyt Accompanying blog post: / 5810d35d54b4 0:00 Shannon Entropy and Information Gain 2:22 What ball will we pick? 4:33 Quiz 5:06 Question 5:14 Game 7:17 Probability of Winning 7:45 Products 11:00 What if there are more classes? 12:34 Sequence 2 13:44 Sequence 3 14:57 Naive Approach 15:34 Sequence 1 19:44 General Formula
▶︎
The Key Equation Behind Probability
▶︎
A friendly introduction to Bayes Theorem and Hidden Markov Models
▶︎
The Story of Information Theory: from Morse to Shannon to ENTROPY
▶︎
The most beautiful formula not enough people understand
▶︎
A Short Introduction to Entropy, Cross-Entropy and KL-Divergence
▶︎
You've (Likely) Been Playing The Game of Life Wrong
▶︎
Claude Shannon at MIT: The best master's thesis in history | Neil Gershenfeld and Lex Fridman
▶︎
Conan O’Brien Delivers the Commencement Address | Harvard Commencement 2026
▶︎
Entropy in Compression - Computerphile
▶︎
I Hacked This Temu Router. What I Found Should Be Illegal.
▶︎
Naive Bayes classifier: A friendly approach
▶︎
The Strange Math That Predicts (Almost) Anything
▶︎
The Pattern Nobody Can Prove (But Everyone Believes)
▶︎
"A.I. and Our Economic Future," Professor Chad Jones
▶︎
Understanding Shannon entropy: (1) variability within a distribution
▶︎
The Beta distribution in 12 minutes!
▶︎
When Math Isn’t Based in Reality
▶︎
We finally understood orbital shapes intuitively! (My mind is blown)
▶︎
Information Theory Basics
▶︎