Local LLM performance tuning

Let's take a look at our dual-GPU local LLM setup and see if we can tune the settings to squeeze a few more tokens per second out of it! We'll talk about the advantages and disadvantages of MTP (multi-token prediction), and demonstrate what changing a few startup options to llama-server can do! 00:00 Intro 00:49 Baseline reading 01:43 Layer split 03:28 Tensor split 04:16 Prompt prefill vs generation 05:02 MTP 06:46 Enabling MTP 08:01 What's the catch? 09:55 The tradeoff 10:34 Conclusion