Build a Fast Local Coding Agent on a Budget GPU with Llama.cpp, REAP and Pi

Build a powerful local coding agent on a budget GPU — a used $250 RTX 3060 running Llama.cpp and Pi, fully private, with no API key, no subscription, and no rate limits. The trick is prefill: the one number nobody tunes. Tune it and the same card jumps from 205 to 1,142 prefill tokens/sec. Same hardware, same model, one flag — 5.5× faster. Here's the part most people miss: a coding agent reads your code ~20× more than it writes. That read phase — prefill — is the part a GPU can batch. Compress the KV-cache, set the micro-batch, and the whole long context stays resident in 12GB of VRAM. The card stops idling. The waiting goes away. The full stack, all real, all free except the card: • Hardware — used RTX 3060, 12GB VRAM (~$250 second-hand) • Engine — Llama.cpp + TurboQuant (quantization that keeps a long-context model resident in VRAM) • Model — a REAP-pruned mixture-of-experts (pruned Qwen or GLM): 50% of the experts removed, near-lossless on code • Agent — Pi, the coding loop that feels like Claude Code (read → plan → edit → test → repeat) • Remote — Tailscale, so you drive the rig from your laptop or phone, anywhere Honest limits, because the pitch only works if it's credible: 12GB has a ceiling — the biggest frontier models still want the cloud. REAP trades a sliver of off-code performance for the size win. And first setup is an afternoon, not a one-click app. But if you want a private agent that reads your code all day, never leaves your machine, and never sends you a bill — nothing beats it. Chapters: 00:00 The $250 number that shouldn't be possible 01:13 Why your coding agent feels slow 02:15 The lever nobody tunes (prefill) 03:21 The $250 machine 04:05 The exact 5-piece stack 07:08 Watch it run 08:38 What you actually bought