How to Calibrate Media Mix Models: Bayesian Priors, ROAS, and Lift Tests | PyMC Labs

Why do media mix models get it wrong - and what can you do about it? In this webinar, four principal data scientists from PyMC Labs break down the core calibration techniques that make MMMs actually useful for decision-making. We start with the fundamentals: why MMMs are causal inference problems at their core, and why ignoring that leads to bad budget decisions. From there, we walk through prior predictive checks, ROAS parametrization (based on a 2024 Google Research paper), and a novel approach we developed at PyMC Labs - additional saturation likelihoods that use lift test data directly in the model graph. We also cover geo-level media mix models, how partial pooling lets you share information across regions without testing every single geo, and the real-world limitations of lift test calibration that nobody talks about enough. Everything shown in this session is available in our open-source libraries, PyMC-Marketing and CausalPy. You can run these techniques on your own data today. Speakers: Juan Orduz, Principal Data Scientist, PyMC Labs Ben Vincent, Principal Data Scientist, PyMC Labs William Dean, Principal Data Scientist, PyMC Labs Carlos Trujillo, Principal Data Scientist, PyMC Labs Resources: PyMC-Marketing Docs: Prior Predictive Modeling: https://www.pymc-marketing.io/en/stab... Calibration via ROAS Parametrization: https://www.pymc-marketing.io/en/stab... Calibration via Additional Saturation Likelihood: https://www.pymc-marketing.io/en/stab... Geo-level MMM Calibration: https://www.pymc-marketing.io/en/stab... Other Resources: Media Mix Model Calibration With Bayesian Priors: https://research.google/pubs/media-mi... Media Mix Model and Experimental Calibration: A Simulation Study: https://juanitorduz.github.io/mmm_roas/ Lift Testing with Interrupted Time Series: A Marketing Case Study: https://causalpy.readthedocs.io/en/st... Why you need more than one lift test per channel: https://drbenvincent.github.io/posts/... Contact us for consulting and collaborations: https://www.pymc-labs.com/contact Join our Discord community:   / discord   PyMC Labs website: https://www.pymc-labs.com/ Email: [email protected] Chapters: 0:00 Introduction and Agenda 1:20 Why Calibrate Media Mix Models? 3:29 The Causal Inference Challenge in MMMs 6:41 Prior Predictive Modeling 10:59 Simulated Example: How Bias Breaks Your Model 14:54 ROAS Parametrization Technique 18:57 Saturation Likelihood Calibration with Lift Tests 23:08 Cross-Validation and the Value of Adding Experiments 27:34 Geo-Level MMMs and Partial Pooling 35:56 What Lift Test Calibration Cannot Do 42:31 Operational Challenges of Running Experiments 45:52 Key Takeaways and Final Remarks 50:13 Q&A: Audience Questions