AI Benchmarks Are Broken — So I Built One You Can Play

AI benchmarks are broken — so I built one you can actually play. A hands-on bench that runs Fable 5, GPT-5.6-Sol, and Grok 4.5 through the same prompt so you can touch the result, not just read a bar chart. SWE-bench, Humanity's Last Exam, math suites — they capture narrow slices of what a model can do, and they get gamed (Grok 4.5 reportedly had Cursor Bench answers baked into its pre-training data). A green bar tells you a model solved one specific problem. It tells you nothing about intent, taste, or how it synthesizes interdisciplinary understanding into something a human can actually feel. So I made the Jonny bench at getatrium.dev/bench: a growing exhibition where each model one-shots the same short prompt and you play what it built. This round it's a playable 3D Gradius in Three.js — same six-line prompt, wildly different results from Fable 5, GPT-5.6-Sol, and Grok 4.5, each with wall time, tokens, cost estimate, and effort level shown. New builds roughly weekly. The whole thing runs inside atrium, the dev environment I'm building for AI coding agents. This isn't meant to replace real benchmarks — they matter for tracking how models trend. It's meant to give you a way to develop your own intuition instead of trusting a screenshot on X. ⏱ Chapters 0:00 Fable 5 one-shots a playable 3D Gradius 1:03 Why benchmarks don't tell the whole story 1:20 My hot take: AI benchmarks are broken 2:35 When benchmarks get contaminated (Grok 4.5) 5:07 Building my own hands-on bench 5:40 The Jonny bench at getatrium.dev/bench 7:06 GPT-5.6-Sol & Grok 4.5, side by side 8:02 Where benchmarks still matter 10:02 Touch and feel a model's real intelligence 🔗 Links Play the bench: https://getatrium.dev/bench Try atrium: https://getatrium.dev ▶ Got a prompt I should run through the bench? Drop it in the comments. #AIbenchmarks #AIcoding #Fable5 atrium is a resumable development environment for AI coding agents. → https://getatrium.dev