How to Succeed in Vertical AI

Manual workflows in specialized industries still consume enormous effort and face complex challenges when implementing AI. What if the problem isn't the technology, but how we're approaching AI implementation in vertical domains? In this talk, Chris Lovejoy (Head of Clinical AI at Anterior, previously MD from Cambridge) joins us to share lessons learned from building AI agents in verticalized industries, particularly in healthcare, education, recruiting, and retail sectors. We discuss: • Why the "last mile problem" makes it difficult to apply LLMs to specialized industries - moving from demos that work well to production systems that understand specific workflows • How to leverage domain experts to supercharge AI development and why building custom UIs is one of the highest leverage activities to support them • Why prompting beats fine-tuning for verticalized agents in the vast majority of cases, and advanced prompting techniques beyond basic prompt engineering • The challenge of defining "what is good" in specialized contexts where it's not just pass/fail but requires domain expert evaluation • Real-world strategies: intelligent performance monitoring, building secure LLM-native architecture, and extracting failure modes from production outputs • Why understanding your existing data and processes through domain expert review is more critical than chasing the latest model benchmarks • Building and maintaining customer trust through systematic incorporation of domain expertise Chris shares insights from scaling AI across multiple verticals, revealing why creating systems that continuously incorporate domain knowledge and minimize friction in the expert review process is more important than focusing solely on model sophistication. The discussion covers practical strategies for building evaluation workflows, managing the accuracy-latency tradeoff, handling information retrieval in RAG systems, and creating flywheels that systematically improve probabilistic AI applications. About Anterior: https://www.anterior.com/ Connect with Chris: LinkedIn:   / dr-christopher-lovejoy   X/Twitter: https://x.com/ChrisLovejoy_ TIME STAMPS 0:00 Introduction and Overview 01:29 Meet Chris: Background and Experience 01:43 Supercharging AI with Domain Experts 14:11 Prompting vs Fine-Tuning: Best Practices 19:57 Building Customer Trust in AI 25:23 AI Confidence and Data Handling 25:55 Strategies for Using Customer Data 27:36 Isolated Environments and Synthetic Data 28:56 Security Considerations for LLMs 30:43 Hiring Domain Experts 35:15 Q&A Session If you want to learn more about improving rag applications check out: https://improvingrag.com/ Stay updated: X/Twitter: https://x.com/jxnl LinkedIn:   / jxnlco   Site: https://jxnl.co/ Newsletter: https://subscribe.jxnl.co/