End to End Modern Distributed Data Lakehouse using Apache Iceberg, Trino, Airflow, DBT and Minio

FULL VIDEO AVAILABLE HERE:    • End to End Modern Distributed Data Lakehou...   In this hands-on project, we build a modern distributed data lakehouse from scratch using cutting-edge open-source technologies: Apache Iceberg, Trino, Airflow, DBT, MinIO, and Project Nessie. This end-to-end data engineering project takes you through every stage of designing, setting up, and running a fully functional data lakehouse with practical insights into distributed systems, data pipeline orchestration, query optimization, and modern data lakehouse best practices. What you’ll learn in this project: ✅ High-level architecture of a modern data lakehouse ✅ Apache Iceberg fundamentals and setup ✅ Distributed Trino cluster (master-worker architecture) ✅ Data orchestration with Apache Airflow ✅ Transformations with DBT (Medallion Architecture) ✅ Object storage integration with MinIO ✅ Versioned metadata with Project Nessie ✅ Trino query optimization and performance tuning If you want to master data lakehouses and modern data engineering, this project is a must-watch. 👉 Don’t forget to like, share, and subscribe for more end-to-end data engineering projects! Timestamps: 0:00 Introduction 1:02 High Level System Architecture Walkthrough 12:00 What is Apache Iceberg? 24:00 Setting up Distributed Data Lakehouse from Scratch 40:15 Apache Airflow DAG Pipeline 1:46:10 DBT Project Setup with Medallion Architecture 1:30:55 Trino Cluster Optimisation 1:32:37 Trino Query Engine with DataGrip 1:41:17 Results and Discussion 1:55:00 Outro Resources: Full Source Code: https://buymeacoffee.com/yusuf.ganiyu... Hashtags 🌟 #ApacheIceberg #Trino #Airflow #DBT #MinIO #DataEngineering #MordernDataLakehouse #BigData #DistributedSystems #ETL