Data: The Land DevOps Forgot • Michael Nygard • GOTO 2024

This presentation was recorded at GOTO Copenhagen 2024. #GOTOcon #GOTOcph https://gotocph.com Michael Nygard - General Manager of Data at Nubank ‪@michaelnygard3079‬ RESOURCES   / mtnygard     / mtnygard   http://www.michaelnygard.com Links https://martinfowler.com/articles/dat... ABSTRACT We've transformed our operational systems to microservices with stream-aligned teams that own what they build. We know that working in short cycles with fast feedback produces better value sooner, safer, and happier. But there's a dark underbelly. Those microservice databases get pulled out into analytics processing that runs in monolithic batches that run for hours on hundreds of cores, then break with an error. Feedback takes days, and the centralized data team can't keep up with the pace of the business or data growth. There's another way, gaining traction. Data mesh promises to re-decentralize data processing along with ownership. It applies product thinking to internal data products. We will talk about moving to data mesh: the promise and the pitfalls, what you can buy and what you'll need to build, and how to get internal buy-in for the transformation. [...] TIMECODES 00:00 Intro 02:45 Operational vs analytic 07:28 The opposite of DevOps 08:07 New paradigm required 09:35 Data Mesh 18:50 Supporting the data owner 22:10 Platforms create incentives 28:55 Make the need visceral 31:17 Summary 33:07 Outro Download slides and read the full abstract here: https://gotocph.com/2024/sessions/3338 RECOMMENDED BOOKS Michael Nygard • Release It! 2nd Edition • https://amzn.to/3WJeKV8 Michael Nygard • Release It! 1st Edition • https://amzn.to/3XCkiRf Zhamak Dehghani • Data Mesh • https://amzn.to/3tTCwAC Eberhard Wolff & Hanna Prinz • Service Mesh • https://leanpub.com/service-mesh-primer Piethein Strengholt • Data Management at Scale • https://amzn.to/3tya08H Martin Kleppmann • Designing Data-Intensive Applications • https://amzn.to/3mk2Roj Sandeep Uttamchandani • The Self-Service Data Roadmap • https://amzn.to/3wAw5W2 https://bsky.app/profile/gotocon.com   / gotocon     / goto-     / goto_con     / gotoconferences   #DataMesh #Data #Observability #Automation #Microservices #SoftwareArchitecture #DevOps #MicroservicesOfData #Decentralize #SLO #MichaelNygard CHANNEL MEMBERSHIP BONUS Join this channel to get early access to videos & other perks:    / @goto-   Looking for a unique learning experience? Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech Sign up for updates and specials at https://gotopia.tech/newsletter SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily. https://www.youtube.com/user/GotoConf...