Под капотом каталога данных — Анастасия Ожигина, Тинькофф

At the recent HighLoad++ Foundation 2022 conference, we presented the creation of Data Detective (DD), Tinkoff's proprietary data catalog. We didn't have enough time to cover everything we wanted. It was important for us to make DD as flexible as possible for all types of metadata with detailed lineage. The question arose: how to achieve versatility in the tool? Anastasia explained how DD works "under the hood" and how we turned our dreams and aspirations into reality. We'll discuss in detail: — the data model of our catalog, which enabled us to achieve flexibility; — the product architecture and its evolution; — the challenges we faced and how to address them. This talk will be useful for those who have already implemented a data catalog or are considering developing and implementing this tool. Digests, articles, and meetup announcements: https://t.me/itstinkoff Blog on Habr: https://l.tinkoff.ru/habr-tinkoff Our vacancies: https://l.tinkoff.ru/its-career