Building a High Performance Real-Time Analytics Database - End to End Data Engineering Project

In this video you will be building a High Performance Real-time Analytics Database using state of the art tools in the Apache Ecosystem like Apache Kafka, Apache Druid, Apache Superset, Docker, and Orbstack. FOR MORE DATA ENGINEERING COURSES: datamasterylab.com 📚 What You'll Learn: 👉 Understand Apache Frameworks for Data Engineering 👉 Streaming data into Apache Kafka 👉 Using Zookeeper for distributed synchronization 👉 Data processing with Apache Druid 👉 Data storage and Realtime Aggregations with Apache Druid 👉 Containerising your data engineering environment with Docker ✨ Timestamps: ✨ 0:00 Introduction 1:47 List of Apache Frameworks for Data Engineering 3:20 System Architecture 10:36 Starting up a project from scratch 13:18 Setting up the containers and services on Docker 28:10 Streaming data into Apache Kafka 42:35 Apache Druid Walkthrough 48:57 Connecting Apache Druid to Apache Kafka 1:00:04 Realtime Queries and Aggregations on Apache Druid 1:07:34 Time Aggregations on Apache Druid 1:09:32 Outro 👦🏻 My Linkedin:   / yusuf-ganiyu-b90140107   🚀 Twitter:   / yusufoganiyu   📝 Medium:   / yusuf.ganiyu   🌟 Please LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟 Like this video? Buy me a coffee ❤️ https://www.buymeacoffee.com/yusuf.ga... 🔗 Useful Links and Resources: 👉 Full Source Code: buymeacoffee.com/yusuf.ganiyu/full-source-code-building-high-performance-realtime-analytics-database 👉 Apache Druid Documentation: https://druid.apache.org/docs/latest/... 👉 Apache Superset Documentation: https://superset.apache.org/docs/ 👉 Docker Documentation: https://docs.docker.com/ 👉 Orbstack Documentation: https://orbstack.dev/docs ✨ Tags ✨ Data Engineering, Apache Kafka, Apache Druid, Real-time Analytics, Apache Superset, Docker, Orbstack, High-Performance Databases, Big Data, Streaming Data, Data Processing, Open Source, Data Aggregation ✨ Hashtags ✨ #DataEngineering #ApacheKafka #ApacheDruid #RealTimeAnalytics #ApacheSuperset #Docker #BigData #DataScience #StreamingData #OpenSource #DataAggregation #Orbstack #HighPerformanceDatabases #bigdatatechnologies

Smart City End to End Realtime Data Engineering Project | Get Hired as an AWS Data Engineer
▶︎

Smart City End to End Realtime Data Engineering Project | Get Hired as an AWS Data Engineer

TypeScript in Express – TypeScript Tutorial
▶︎

TypeScript in Express – TypeScript Tutorial

How to Back Up PostgreSQL in Docker (Local, S3, and Plakar)
▶︎

How to Back Up PostgreSQL in Docker (Local, S3, and Plakar)

[ 37hica1 h@ckin9 ] 🎫🧪⛓️ HackTheBox - Orion : 3-Combo Attack 👊👊👊
▶︎

[ 37hica1 h@ckin9 ] 🎫🧪⛓️ HackTheBox - Orion : 3-Combo Attack 👊👊👊

DataFusion Community Showcase Vol. 2: DataFusion Comet & DataFusion Ballista
▶︎

DataFusion Community Showcase Vol. 2: DataFusion Comet & DataFusion Ballista

Automating Data Collection Pipelines for Finance and Stock Market Analytics
▶︎

Automating Data Collection Pipelines for Finance and Stock Market Analytics

PySpark Full Course [2024] |  Learn PySpark | PySpark Tutorial | Edureka
▶︎

PySpark Full Course [2024] | Learn PySpark | PySpark Tutorial | Edureka

Tibetan Healing Sounds  for Mind, Body & Soul | Relaxation, Sleep & Inner Peace
▶︎

Tibetan Healing Sounds for Mind, Body & Soul | Relaxation, Sleep & Inner Peace

معمّم شيعي يقول لرامي عيسى: "خصمك الإمام علي يوم القيامة!" 😱 لكن ما حدث بعدها صدم الجميع!🔥
▶︎

معمّم شيعي يقول لرامي عيسى: "خصمك الإمام علي يوم القيامة!" 😱 لكن ما حدث بعدها صدم الجميع!🔥

Is This Wish Meant to Be Fulfilled? 🧚🤲 Detailed Pick a Card Tarot Reading ✫・
▶︎

Is This Wish Meant to Be Fulfilled? 🧚🤲 Detailed Pick a Card Tarot Reading ✫・

Lawrence Wilkerson: Iran-Krieg spitzt sich zur globalen Krise zu
▶︎

Lawrence Wilkerson: Iran-Krieg spitzt sich zur globalen Krise zu

The PROBLEM with Capitalism - Smarter Every Day 316
▶︎

The PROBLEM with Capitalism - Smarter Every Day 316

RESTORE YOUR PRAYER ALTAR BEFORE IT'S TOO LATE | Apostle Joshua Selman. #apostlejoshuaselman
▶︎

RESTORE YOUR PRAYER ALTAR BEFORE IT'S TOO LATE | Apostle Joshua Selman. #apostlejoshuaselman

1.2 Billion Records Per Hour High Performance Kafka and Spark - End to End Data Engineering Project
▶︎

1.2 Billion Records Per Hour High Performance Kafka and Spark - End to End Data Engineering Project

Heals You While You Sleep (No Ads) • Relieves Stress • Increases Deep Sleep
▶︎

Heals You While You Sleep (No Ads) • Relieves Stress • Increases Deep Sleep

Michael Hudson: Der US-Plan zur Wiederbelebung der geoökonomischen Dominanz
▶︎

Michael Hudson: Der US-Plan zur Wiederbelebung der geoökonomischen Dominanz

After an Hour and a Half of Debate: A Saudi TikToker and Sheikh Zain Khairallah.. What Happened i...
▶︎

After an Hour and a Half of Debate: A Saudi TikToker and Sheikh Zain Khairallah.. What Happened i...

Data Engineering Course for Beginners
▶︎

Data Engineering Course for Beginners

Jon Stewart on Trump's "Meritocracy" & Desi Lydic on MAGA Moving Iran Goalposts | The Daily Show
▶︎

Jon Stewart on Trump's "Meritocracy" & Desi Lydic on MAGA Moving Iran Goalposts | The Daily Show

The Most Powerful Manifestation Technique ... It Works So Fast It's Scary.
▶︎

The Most Powerful Manifestation Technique ... It Works So Fast It's Scary.