🚫 Applying a Causal Attention Mask – Live Coding with Sebastian Raschka (Chapter 3.5.1)

Check out Sebastian Raschka's book 📖 Build a Large Language Model (From Scratch) | https://hubs.la/Q03l0mSf0 📖 In this practical coding session, acclaimed ML author ‪@SebastianRaschka‬ demonstrates how to implement a causal attention mask — a vital component in autoregressive transformer models. Covering Chapter 3.5.1 of his book Build a Large Language Model (From Scratch), this video shows how to ensure that a token only attends to itself and earlier tokens during training. 0:00 - Introduction 0:58 - Causal Attention Mask Explained 3:01 - Implementing a Causal Self-Attention Class 7:16 - Simplifying Mask Application 11:11 - Preview of Upcoming Content 📘 About the Book Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself! 💡 Ideal for ML engineers, NLP researchers, and developers building generative language models. 🔗 Get the Book: https://hubs.la/Q03l0mSf0 📺 Subscribe for expert-led walkthroughs, transformer tutorials, and live-coding sessions. #SebastianRaschka #LLM #CausalMasking #AttentionMask #Transformers #DeepLearning #NLP #ManningPublications #LiveCoding

Join Today
🛡️ Masking Additional Attention Weights with Dropout – Live Coding w/ Sebastian Raschka (Ch. 3.5.2)
▶︎

🛡️ Masking Additional Attention Weights with Dropout – Live Coding w/ Sebastian Raschka (Ch. 3.5.2)

Understanding causal attention or masked self attention | Transformers for vision series
▶︎

Understanding causal attention or masked self attention | Transformers for vision series

Yann LeCun's $1B Bet Against LLMs [Part 1]
▶︎

Yann LeCun's $1B Bet Against LLMs [Part 1]

✅ Evaluating the Fine-Tuned LLM – Live Coding with Sebastian Raschka (Chapter 7.8)
▶︎

✅ Evaluating the Fine-Tuned LLM – Live Coding with Sebastian Raschka (Chapter 7.8)

Using Large Language Models | Build Your Own LLM Workshop #1
▶︎

Using Large Language Models | Build Your Own LLM Workshop #1

System Design Explained: APIs, Databases, Caching, CDNs, Load Balancing & Production Infra
▶︎

System Design Explained: APIs, Databases, Caching, CDNs, Load Balancing & Production Infra

Transformers, the tech behind LLMs | Deep Learning Chapter 5
▶︎

Transformers, the tech behind LLMs | Deep Learning Chapter 5

Visualizing transformers and attention | Talk for TNG Big Tech Day '24
▶︎

Visualizing transformers and attention | Talk for TNG Big Tech Day '24

🧱 Stacking Multiple Single-Head Attention Layers – Live Coding w Sebastian Raschka (Ch. 3.6.1)
▶︎

🧱 Stacking Multiple Single-Head Attention Layers – Live Coding w Sebastian Raschka (Ch. 3.6.1)

What to teach when AI writes the code | Rainer Stropek | TEDxLinz
▶︎

What to teach when AI writes the code | Rainer Stropek | TEDxLinz

The Strange Math That Predicts (Almost) Anything
▶︎

The Strange Math That Predicts (Almost) Anything

Don't learn AI Agents without Learning these Fundamentals
▶︎

Don't learn AI Agents without Learning these Fundamentals

1: Introduction to Neural Networks and Deep Learning; Training Deep NNs
▶︎

1: Introduction to Neural Networks and Deep Learning; Training Deep NNs

Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)
▶︎

Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)

Attention in transformers, step-by-step | Deep Learning Chapter 6
▶︎

Attention in transformers, step-by-step | Deep Learning Chapter 6

How to Measure LLM Confidence: Logprobs & Structured Output
▶︎

How to Measure LLM Confidence: Logprobs & Structured Output

PINK & ORANGE GRADIENT IN HD [3 HOURS]
▶︎

PINK & ORANGE GRADIENT IN HD [3 HOURS]

But what is a neural network? | Deep learning chapter 1
▶︎

But what is a neural network? | Deep learning chapter 1

Coding a Transformer from scratch on PyTorch, with full explanation, training and inference.
▶︎

Coding a Transformer from scratch on PyTorch, with full explanation, training and inference.