Perfect training data cripples reasoning — RLVR vs SFT has a provable exponential gap

Everyone assumes clean, flawless examples are the best reasoning data — and a new theory paper proves that intuition is backwards. By formalizing reasoning as path-finding through a maze, two researchers show imitation learning provably can't teach backtracking, while reinforcement learning learns it for free from the model's own failures. The result is a clean, exponential gap that reframes what 'high-quality reasoning data' even means. Full episode page: https://paperdive.ai/episodes/163-provable... Paper: Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently Authors: Wei, Kim Read the paper: https://arxiv.org/abs/2606.22938 What you'll take away: Why training on clean, backtracking-free solutions provably freezes a model's ability to retreat from dead ends — there's no gradient signal where there's no data How modeling reasoning as path-finding through a maze turns 'backtracking' into something you can prove theorems about The headline result: RL scales linearly with reasoning depth (W·K) while imitation blows up exponentially (W·L^K), from the identical starting model Why bolting a clever search wrapper onto a weak imitation model helps a lot but still can't fully close the gap The steelman critique: the central theorem is close to true by construction, and the exponential drama leans on a chosen graph topology and a deliberately pessimistic definition of SFT The practical payoff — why distilling from an RL-trained model works precisely because you inherit its messy recoveries, not just its answers Chapters: 0:00 Is clean data secretly the problem? 1:40 Two ways to train, one key difference 3:53 Turning reasoning into a maze 6:16 No examples, no nudge 9:15 Linear versus falling off a cliff 10:27 How RL escapes the trap 13:24 Does it survive a real algorithm? 15:31 How true by construction is this? 18:51 The dead ends are the curriculum This episode is AI-generated. The script was written by an AI language model and the host voices were synthesized by Eleven Labs. The producer is not affiliated with Anthropic or Eleven Labs. The on-screen illustrations were generated by OpenAI GPT Image.