Deep Dive Into The Toolformer

This week we cover the "Toolformer: Language Models Can Teach Themselves to Use Tools" paper from Meta and Universitat Pompeu Fabra. This paper shows how you can train your LLM to use tools like a calculator and calendar through API calls. -- Get Oxen 🐂 https://oxen.ai/ Oxen.ai makes versioning your datasets as easy as versioning your code! Even is millions of unstructured images, we quickly handle any type of data so you can build cutting-edge AI. -- Toolformer 📜 https://arxiv.org/abs/2302.04761 The Datasets 🔢 https://www.oxen.ai/Laurence/mlqa https://www.oxen.ai/Laurence/lama https://www.oxen.ai/Laurence/ASDiv https://www.oxen.ai/Laurence/SVAMP https://www.oxen.ai/Laurence/web_ques... https://www.oxen.ai/Laurence/MAWPS https://www.oxen.ai/Laurence/templama https://www.oxen.ai/datasets/OxenAI-P... Filtering Functions ✂️ https://github.com/lucidrains/toolfor... Toolformer Notes 📜 https://www.oxen.ai/blog/toolformer-l... Join Arxiv Dives 🤿 https://oxen.ai/community Discord 🗿   / discord   -- Chapters 0:00 Intro to the Toolformer 6:40 Toolformer Architecture 7:43 Approach 9:39 Creating the Training Data 12:24 Generate API Call Data 13:36 Together AI Demo 15:35 Axiv Paper Examples 18:00 Execute API Calls 19:53 Filtering API Calls and Math 31:15 Experiments 32:12 Results 34:14 Scaling Laws 35:22 Questions