SLMs - When and When NOT to use them (+ Mistral 3.1 & Gemma-3 Bakeoff)

Datasets & Slides 📝 https://www.oxen.ai/ox/Arxiv-Dive-Smo... https://www.oxen.ai/ox/SmolLMs https://www.oxen.ai/ox/mbrp https://www.oxen.ai/ox/SimpleQA Join Arxiv Dives 🤿 https://oxen.ai/community Discord 🗿   / discord   Use Oxen AI 🐂 https://oxen.ai/ Oxen AI makes versioning your datasets as easy as versioning your code! Even is millions of unstructured images, the tool quickly handles any type of data so you can build cutting-edge AI. -- Chapters 0:00 Welcome to Arxiv Dive 1:12 $ whois 1:59 $ whoami 3:18 What is Oxen.ai 4:24 Lets Talk Smol Lms4:35 Benefits of Smol Lms 6:33 When Not to Use Smol LMs 7:47 What is a Data Flywheel 9:01 Why Smol LMs Are Important Now 13:42 Did I Use a Framework for SFT or RL 14:09 Only Your Data and Criteria Matters 16:18 Gemma-3 vs. Mistral-3.1 Evals 16:41 How to Evaluate a Model 26:49 o3-mini, Mistral Small-3.1, and Gemma-3 on SimpleQA 28:17 Training a Model to Program in Rust 34:45 o3-mini, Mistral Small-3.1, and Gemma-3’s Eval on Rust 38:17 Questions 43:36 What About Smol Multimodal Models? 48:56 Test a Homemade Phi-4 Multimodal Chatbot 58:45 QR Code for Free Compute Credits