Bielik LM in Triton - Can I Actually Pull This Off?

First episode of the series on implementing Polish language model Bielik 1.5 (1.6B parameters) from scratch using GPU kernels in Triton! In this episode: Bielik 1.5 Instruct architecture Grouped Query Attention (GQA) SwiGLU activation and RMSNorm Introduction to GPU programming in Triton Plan for the entire series (8 episodes) Timestamps 0:00 - Introduction: Building an LLM from scratch 1:06 - What is Bielik? (Polish Language Model) 1:46 - High-Level Architecture Overview 2:25 - Grouped Query Attention (GQA) vs. Multi-Head Attention 3:20 - Why OpenAI Triton? (Triton vs. CUDA vs. PyTorch) 4:28 - The Roadmap: Flash Attention, RoPE, and Kernels 5:04 - Summary & What’s Next Resources & Links: Code Repository (GitHub): https://github.com/qooba/bielik-anato... Bielik Model (HuggingFace): https://huggingface.co/speakleash/Bie... SpeakLeash Community: https://speakleash.org/en/speakleash-... OpenAI Triton Documentation: https://triton-lang.org/main/index.html #bielik #llm #gpu #triton #machinelearning #polish #ai #transformer #deeplearning