The Math That Shrinks ChatGPT Onto a Laptop

How does a 326GB language model become something you can run locally on a laptop? In this Nerdy Dives episode, we break down the engineering behind local LLMs: quantization, memory bandwidth, GGUF, llama.cpp, GPTQ, AWQ, QLoRA, and why your GPU is often waiting on memory instead of doing math. This is the real story behind “AI on your laptop” — not hype, but the papers, numbers, and systems tricks that made it possible. We cover: • Why LLM inference is usually memory-bound, not compute-bound • How FP16 weights turn 70B models into 140GB monsters • Why naive quantization breaks at scale • The 0.1% outlier features that can destroy a model if handled wrong • How LLM.int8(), GPTQ, and AWQ made 8-bit and 4-bit models practical • Why GGUF and llama.cpp became the foundation of local AI • How QLoRA made finetuning huge models possible on one GPU • Why MoE and distillation shrink the work even further If you’ve used Ollama, LM Studio, llama.cpp, GGUF models, or local AI tools and wondered how this stuff actually works, this is the deep dive. Sources include: LLM.int8(), GPTQ, AWQ, QLoRA, llama.cpp, GGUF docs, OpenAI gpt-oss, and knowledge distillation papers. Nothing in this video is sponsored. All models shown were run locally.