Gilles Gignac | Intelligence, Benchmarks & "Artificial Achievement" | Podcast #20

Are today's AI systems truly intelligent, or are they doing something different — what my guest calls "artificial achievement"? I'm joined by Gilles Gignac, associate professor of psychology at the University of Western Australia in Perth, whose research sits right at the intersection of human intelligence and AI measurement. We get into how to actually define intelligence across humans and machines, why the benchmarks used to test LLMs are often broken, what the "positive manifold" and general intelligence tell us about both, and the role of theory of mind, metacognition, world models, and the much-misused Dunning-Kruger effect. A deep, careful conversation on what these systems can and can't do. Gilles Gignac — associate professor of psychology, University of Western Australia (Perth). Researches intelligence, psychometrics and AI measurement, plus metacognition, narcissism, financial literacy and personality. 0:00 Is AI really intelligent — or just "artificial achievement"? 0:50 Defining intelligence vs. achievement (Gignac & Szodorai, 2024) 4:35 Does the "substrate" matter? Toward a universal definition (Legg & Hutter) 9:25 Why humans are such efficient learners — and why IQ tests ignore learning 13:35 Generality, the "positive manifold," and what "AGI" really means 18:25 Post-training & specialisation: Claude's coding vs. OpenAI's breadth 20:35 The Matthew Effect: why knowledge compounds in humans 22:50 Pattern recognition: are humans and LLMs doing the same thing? 26:20 What's at the heart of "g"? Mixture of experts & a theory of intelligence 31:20 Does general intelligence really exist? Latent "g" vs. network models 37:10 What AI benchmarks get wrong (contamination, length, bad items) 43:15 A better test: why you only need ~60 good items 46:00 Testing reasoning with truly novel items (the new study) 51:20 The "worst performance rule" & catastrophic AI errors (glue on pizza) 58:20 Theory of mind, metacognition & job performance — what IQ misses 1:04:30 When LLMs can't "go off script" (the counting-names problem) 1:07:20 World models & whether LLMs can get us there (Yann LeCun) 1:11:45 Augmenting vs. replacing human cognition (sycophancy & gamed leaderboards) 1:15:20 Is the Dunning-Kruger effect overblown?