Code Memory Made This Agent Dumber — Here's Why (Metis Deep Dive)

An AI agent that distilled its hard-won experience into reusable code scored ten points worse than an agent with no memory at all. This episode unpacks why the sophisticated-looking move — freezing lessons into callable tools — is also the fragile one, and what the right fix turns out to be. You'll come away understanding the single most basic decision in building agents that learn on the job: when a lesson should stay as soft advice, and when it's earned the right to become code. Full episode page: https://paperdive.ai/episodes/168-metis-br... Paper: Metis: Bridging Text and Code Memory for Self-Evolving Agents Authors: Dai, He, Li et al. Read the paper: https://arxiv.org/abs/2606.24151 What you'll take away: Why storing an agent's experience as callable code can drop it below an agent with no memory at all — a 22-point collapse the moment it has to generalize The 'injection asymmetry': text is consumed as adaptable advice you filter through reality, while code is a trusted black box whose flaws propagate to every caller and suppress the agent's own recovery behavior Metis's 'text first, code earned' policy — sorting experience into plans, facts, and pitfalls, and crystallizing only recurring plans into tools using the desire-path principle Why the codifier deliberately never reads the messy trajectory, building tools from the clean query pattern instead — and how that lets even failed runs safely count toward codification The ablation that proves the recurrence gate: an 'Eager' version cost 47% more to build, scored worse, and left over half its tools never invoked Where the clean story has a seam: the headline result is really about ungated, trajectory-trained, unvalidated code on a single benchmark — not a law that 'code memory is bad' Chapters: 0:00 The brilliant employee with amnesia 3:12 Text advice or a black-box tool? 5:01 The experiment that fixed every variable 8:54 The 22-point collapse 10:17 Why the confident tool fails hard 13:18 Paving only the paths people walk 18:24 Does the machinery actually pay off? 21:52 The seam in the clean story 24:41 Don't pour the concrete too early This episode is AI-generated. The script was written by an AI language model and the host voices were synthesized by Eleven Labs. The producer is not affiliated with Anthropic or Eleven Labs. The on-screen illustrations were generated by OpenAI GPT Image.