Karpathy Bigram explained in 10min..

Check out BlueDot's courses and learn how to help shape the future of AI: http://bluedot.org/calebwritescode Andrej Karpathy's explaination of Bigram Language Model explained Bigram LMs, though simple, it provides powerful insight into the inner mechanics of how tokens are processed in language models. This is a pre-amble for what's next: GPT, which is the 2nd part of the series The Bigram model here incorporates: Tokenization, Vocabulary, Negative Loss Function, Cross Entropy, Logits, SoftMax, Optimizer, and AdamW These are essential ingredients to understand in order to build our knowledge on how LLMs really work as we build our case towards attention and GPT Follow me: X: https://x.com/calebfoundry LinkedIn:   / calebeom   TikTok:   / calebwritescode   Chapters 00:00 Intro 00:41 Tokenization 01:45 Embedding 02:47 Training 03:10 Sponsor: BlueDot 04:15 Batch, Block, Channel 05:35 Update 06:22 Loss 08:40 Backprop, Optimizer 09:03 Result 09:52 Conclusion #karpathy #deeplearning #llm