02. PyTorch From Scratch: Build Your Own GPT (Tensors: The Only Data Type That Matters)

PyTorch From Scratch: Build Your Own GPT — Ep 02 of 19 We build a working GPT from a single tensor up — no magic, no black boxes. Every line of code is on screen and explained: tensors, autograd, an MLP, tokenization, embeddings, self-attention, multi-head attention, the full transformer block, and a tiny GPT you can train and watch generate text. ▶ FULL COURSE CODE (Google Colab — runs in your browser, no setup): https://github.com/kader-xai/ml-cours... Open the PyTorch notebook: https://colab.research.google.com/git... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ EPISODES 01 · Why PyTorch (and What We're Building) 02 · Tensors — The Only Data Type That Matters 03 · Tensor Math & Broadcasting 04 · Autograd — PyTorch Does the Calculus 05 · Your First Trained Thing — Linear Regression by Hand 06 · nn.Module & Optimizers — The Grown-Up Loop 07 · A Real Neural Net — MLP on Real Data 08 · Datasets, DataLoaders & GPU 09 · Text Is the Enemy — Tokenization 10 · Embeddings & Loading a Real Dataset 11 · Position & The Training Batch 12 · Self-Attention From Scratch — Q, K, V 13 · Causal Masking — No Peeking at the Future 14 · Multi-Head Attention 15 · The Transformer Block — Attention + MLP + Residuals 16 · Assembling Our Tiny GPT 17 · Training the Tiny GPT — Loss Falls, Words Appear 18 · Generation & Sampling — Watch It Talk 19 · From Tiny GPT to ChatGPT — What You Built & What's Next ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ MUSIC "Reflections" by Vincent Rubinetti from "The Music of 3Blue1Brown" (2018) https://vincerubinetti.bandcamp.com #PyTorch #DeepLearning #GPT #Transformers #LLM