RAG is Dead: Jeff Huber (Chroma) on Building Agentic Search

Jeff Huber, co-founder and CEO of Chroma, presents the case for retiring RAG as a term and embracing the bitter lesson for context engineering. RAG is dead, Jeff says. The term, not the technique. Ask 25 builders to define RAG, and get 25 different answers. The same goes for "vector database." Both should be banned. What is real, Jeff argues, is that most agent failures are context failures rather than reasoning failures. Context is too much, too little, or confusing. Last year, Chroma published a 45-page research report called Context Rot showing that language model performance is not invariant to context length. The "dumb zone" starts somewhere between 20,000 and 120,000 tokens, depending on the use case. Nobody trusts a million-token context to do anything useful. The way out is to embrace this. You should give agents great tools and good guidance on when and how to use them. Then let them figure out the rest. Context engineering itself, Jeff predicts, will be folded back into the models. Chroma recently shipped a model that does its own lightweight context engineering as a preview of what's coming. The popular path today is the file system — bash tool, sandbox, local files — the way most of us use Codex and Claude Code. Jeff calls file systems bad databases. Concurrency is last-write-wins. There's no indexing. Search is just grep. And a sandbox is a heavyweight tool when you're only reading and writing files. The middle path gaining traction is the virtual file system: bash and files that look real to the agent but are backed by a database. Mintlify recently popularized this approach using Chroma as the backing store, lightweight enough for a read-only documentation use case. Jeff's default recommendation for most use cases is agentic search with hybrid search underneath. Type that into Claude Code, and it will set it up. Chroma Sync, just released, lets you drop files into a bucket and handles the indexing pipeline automatically. Chroma Context One, also new, is an open-source search subagent: an autoregressive language model that takes a query and outputs documents rather than generating them. Search does not benefit from being gigabrained by Claude Opus 4.6 or GPT-5.4. Context One is 10x faster, 25x cheaper, and on par or better than the big models across a battery of public and private search benchmarks. Recorded at the TypeScript AI Demo Day in San Francisco, April 2026, hosted by Mastra. 🔗 JEFF HUBER https://x.com/jeffreyhuber   / jeffchuber   https://jeffhuber.net 📁 CHROMA https://www.trychroma.com https://github.com/chroma-core https://research.trychroma.com 📚 MASTRA RESOURCES https://mastra.ai https://x.com/mastra_ai https://mastra.ai/community/discord https://github.com/mastra-ai https://mastra.ai/course https://mastra.ai/books/principles-of... https://mastra.ai/books/patterns-of-b... WHAT IS MASTRA? Mastra is an open-source TypeScript framework designed for building and shipping AI-powered applications and agents with minimal friction. It supports the full lifecycle of agent development—from prototype to production. You can integrate it with frontend and backend stacks (e.g., React, Next.js, Node) or run agents as standalone services. If you're a JavaScript or TypeScript developer looking to build an agentic or AI-powered product without starting from first principles, Mastra provides the scaffolding, tools, and integrations to accelerate that process. 0:00 Introduction 0:17 RAG is dead — and so is vector database 0:48 Most agent failures are context failures 1:30 Context Rot and the dumb zone 2:59 Embrace the bitter lesson 3:38 Context engineering is folding back into the models 4:09 The file system path and why it's a bad database 5:45 Virtual file systems: Mintlify's approach 7:45 Default recommendation: agentic search with hybrid search 8:15 Chroma Sync 8:28 Chroma Context One: a specialized search subagent 9:59 Q&A: memory, compaction, and distillation 11:24 Q&A: when does agentic search beat RAG