The Wrong AI Conversation: Before AI Improves Your Test, It May Break Your Candidate Pool

The assessment community has done important work grappling with AI, but the conversation may be focused on the wrong risks. While we debate ethical guardrails for using AI for content generation, scoring, and security monitoring, test takers are already being reshaped by AI from the outside in. Rapid and significant disruption is showing up upstream: in who (and how many) choose to enter a field and what the construct looks like by the time they get there. Join us this week as we talk with Jonathan Rubright, founder of Empirically Sound, about the strategic threats assessment programs aren't fully prepared for. We'll cover what happens to validity evidence and cut scores when populations and constructs are shifting faster than governance cycles can track, where the leadership gap is widening in assessment organizations, and what boards and program directors can do now before decisions are made for them. THIS WEEK'S GUEST Jonathan Rubright, Ph.D. is the founder of Empirically Sound, an independent consulting and advisory practice focused on assessment strategy, psychometric leadership, and research. He brings more than 15 years of experience in high-stakes assessment building and leading multidisciplinary teams, including serving as Vice President of Research Strategy at the National Board of Medical Examiners (NBME). His work sits at the intersection of measurement science and organizational strategy, helping programs make programmatic decisions with rigor and clarity. Jonathan advises boards, program leaders, and research teams navigating the moments when technical expertise alone isn't enough. Learn more at empiricallysound.com.