Databricks Full Course for Beginners (2 Hours) - Declarative Pipelines & Lakeflow Designer

Dive into a new way of building data pipelines in Databricks! Instead of writing step-by-step Spark code, you define what you want with SQL and Databricks handles execution, dependencies, and orchestration for you. In this full 2-hour course, you’ll learn how to build end-to-end data pipelines using Spark Declarative Pipelines and Lakeflow Designer. 👉 You’ll build a complete pipeline: ✅ Bronze → Silver → Gold (Medallion architecture) ✅ From raw data to analytics-ready tables ✅ Using SQL instead of complex Spark logic ✅ With automatic dependency handling and orchestration And on top of that: ✅ Build pipelines visually with Lakeflow Designer ✅ Use Genie (Databricks Assistant) to speed up development ✅ Understand how modern Databricks workflows actually work ⏱️ Timestamps 00:03 – Course Overview & What You Will Build 02:56 – Getting Started in Databricks & Dataset Exploration 16:15 – Delta Lake Essentials (ACID, Time Travel, Versioning) 26:06 – Unity Catalog Basics 33:28 – Introduction to Declarative Pipelines & Medallion Architecture 39:02 – Building the Bronze Layer 50:17 – Building the Silver Layer 1:03:33 – Building the Gold Layer 1:11:06 – Scheduling & Running the Pipeline in Lakeflow Designer 1:17:39 – Creating a Pipeline Inside Lakeflow Designer (Part 1) 1:23:15 – Creating a Pipeline Inside Lakeflow Designer (Part 2) 1:33:19 – Streaming Modes in Declarative Pipelines 1:35:57 – Connecting Databricks to an AWS Message Queue 1:41:53 – Building a Simple Declarative Streaming Pipeline 1:56:38 – Summary & Next Steps Resources: ► GitHub Repo for this course: https://github.com/andkret/Databricks... ► Databricks Free Edition: https://bit.ly/4eZHHIg ► All free labs & resources: https://learndataengineering.com/p/fr... Want more? I also did a livestream where I walk through the lab and explain how everything fits together. ► Watch it here: https://youtube.com/live/7lwKcLICy5g?... #dataengineering #dataengineeringcourse #databricks #databrickscourse