The Not So Talked About Reasons Model Monitoring Fails // Oren Razon // MLOps Meetup #88

MLOps Community Meetup #88! Last Wednesday we talked to Oren Razon, Co-founder and CEO of Superwise. //Abstract Congratulations! You’ve researched, developed, and deployed your model. Obviously, the next step is monitoring. Now it’s tempting to focus on the technical and dive into drifts and data anomalies, but there are other critical organizational challenges that can negatively impact your ML operations just as severely. In this talk, we covered both hard organizational challenges like building signal vs noise tolerances and soft organizational challenges like stakeholder identification and aligning expectations. We also shared some best practices on how to lead model observability discovery in your organization and build measurable KPIs for success. // Bio Oren is the co-founder and CEO of Superwise, the leading platform for Model Observability. With over 15 years of experience leading the development, deployment, and scaling of ML products, Oren is an expert ML practitioner specializing in MLOps tools and practices. Previously, Oren managed machine learning activities at Intel’s ML center and operated a machine learning boutique consulting agency helping leading tech companies such as Sisense, Gong, AT&T, and others, to build their machine learning-based products and infrastructure. // Related links ---------- ✌️Connect With Us ✌️------------ Join our Slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, Feature Store, Machine Learning Monitoring and Blogs: https://mlops.community/ Connect with Demetrios on LinkedIn:   / dpbrinkm   Connect with Oren on LinkedIn:   / oren-razon   Timestamps: [00:00] Introduction to Oren Razon [06:41] Superwise Model Observability [07:17] Keyword: Scale [07:37] Model monitoring is hard! [10:57] And at scale it's even harder! [13:25] When are issues real issues? [17:40] Retraining is not always the answer [24:19] Automatic retraining workflow [28:10] Why is a point of view [32:51] Who cares? [36:08] Where do we go from here? [38:12] Models aren't linear [39:07] From monitoring to observability [40:50] Real model observability [42:30] Production first mentality [45:05] Oren's task in Superwise [47:52] Monitoring tooling capacity [50:17] Tips and tricks on monitoring [52:14] Alert hell!