Advanced LLM Post-Training: SFT, DPO, Reinforcement Learning w/ Maxime Labonne (Liquid AI)

In this exclusive guest lecture for the Youth AI Initiative, we hosted Maxime Labonne (Head of Post-Training at Liquid AI & Author of the LLM Engineer's Handbook) for a masterclass on the modern Large Language Model (LLM) training pipeline. Maxime went far beyond the basics; breaking down the exact techniques used by top labs to turn base models into powerful, aligned products. He covered the full stack, from dataset creation to cutting-edge techniques like GRPO. 🚀 What You Will Learn: 📌 Supervised Fine-Tuning (SFT) How to structure instruction data and teach models to follow specific commands. 📌 Preference Alignment (DPO) A deep dive into Direct Preference Optimization and how it aligns models. 📌 Reinforcement Learning (GRPO) & Reasoning Models How new “reasoning models” (like DeepSeek-R1) use Group Relative Policy Optimization (GRPO) to “think,” plan, and verify their chain of thought before answering. 📌 Efficient Training Techniques A comparison of LoRA, QLoRA, and Full Fine-Tuning, including how to train models on limited hardware. 📌 Dataset Curation What makes a dataset truly “good”: accuracy, diversity, complexity, filtering, and balancing. 🌟 About the Youth AI Initiative The Youth AI Initiative is a free, 6-week intensive AI incubator for the brightest high school students. We bridge the gap between academic theory and real-world application through expert-led curriculum and guest insights from leaders at Microsoft, Hugging Face, Liquid AI, and more. 🌐 Learn More: https://youthaiinitiative.com/ 📣 Connect With Us: LinkedIn:   / youth-ai-initiative   Instagram:   / youth_ai_initiative   Twitter: https://x.com/YouthAIInit 🙏 Special Thanks A huge thank you to our main sponsor Tam Finans and our community supporter Global Turks AI. 📍 Timestamps: Introduction 00:00 What is Post-Training 00:53 Supervised Fine-Tuning 05:04 Preference Alignment (DPO) 14:00 Reinforcement Learning (GRPO) 18:10 Conclusion 24:30 Q&A 26:12