AI-First Vulnerability Management: Should CISOs Build or Buy?

Thinking of building your own AI security tool? In this episode, Santiago Castiñeira, CTO of Maze, breaks down the realities of the "Build vs. Buy" debate for AI-first vulnerability management. While building a prototype script is easy, scaling it into a maintainable, audit-proof system is a massive undertaking requiring specialized skills often missing in security teams. The "RAG drug" relies too heavily on Retrieval-Augmented Generation for precise technical data like version numbers, which often fails . The conversation gets into the architecture required for a true AI-first system, moving beyond simple chatbots to complex multi-agent workflows that can reason about context and risk . We also cover the critical importance of rigorous "evals" over "vibe checks" to ensure AI reliability, the hidden costs of LLM inference at scale, and why well-crafted agents might soon be indistinguishable from super-intelligence . Questions asked: 00:00 Introduction 02:00 Who is Santiago Castiñeira? 02:40 What is "AI-First" Vulnerability Management? (Rules vs. Reasoning) 04:55 The "Build vs. Buy" Debate: Can I Just Use ChatGPT? 07:30 The "Bus Factor" Risk of Internal Tools 08:30 Why MCP (Model Context Protocol) Struggles at Scale 10:15 The Architecture of an AI-First Security System 13:45 The Problem with "Vibe Checks": Why You Need Proper Evals 17:20 Where to Start if You Must Build Internally 19:00 The Hidden Need for Data & Software Engineers in Security Teams 21:50 Managing Prompt Drift and Consistency 27:30 The Challenge of Changing LLM Models (Claude vs. Gemini) 30:20 Rethinking Vulnerability Management Metrics in the AI Era 33:30 Surprises in AI Agent Behavior: "Let's Get Back on Topic" 35:30 The Hidden Cost of AI: Token Usage at Scale 37:15 Multi-Agent Governance: Preventing Rogue Agents 41:15 The Future: Semi-Autonomous Security Fleets 45:30 Why RAG Fails for Precise Technical Data (The "RAG Drug") 47:30 How to Evaluate AI Vendors: Is it AI-First or AI-Sprinkled? 50:20 Common Architectural Mistakes: Vibe Evals & Cost Ignorance 56:00 Unpopular Opinion: Well-Crafted Agents vs. Super Intelligence 58:15 Final Questions: Kids, Argentine Steak, and Closing -------------------------------------------------------------------------------- 📱Cloud Security Podcast Social Media📱 _____________________________________ 🛜 Website: https://cloudsecuritypodcast.tv/ 🧑🏾‍💻 Cloud Security Bootcamp - https://www.cloudsecuritybootcamp.com/ ✉️ Cloud Security Newsletter - https://www.cloudsecuritynewsletter.com/ Twitter:   / cloudsecpod   LinkedIn:   / cloud-security-podcast   #cloudsecurity