Advanced Data Quality Use Cases with Airflow and Great Expectations
For this webinar, Benji Lampel (Enterprise Platform Architect @ Astronomer) and Tal Gluck (Software Engineer @ Superconductive) present several Airflow DAGs using Great Expectations that cover advanced DAG patterns and data quality checking cases. Key Takeaways Data Quality comes from the needs of the organization. Learn why data engineers need to be guardians of data and liaisons for quality Data Quality checking in the real world is messy. We will showcase methods to cope with that messiness There’s a strong, documented story of using Airflow and Great Expectations together. This makes it a premier way to add data quality checks into your organization’s data ecosystem.
▶︎
Scaling Out Airflow
▶︎
Always know what to expect from your data with great_expectations
▶︎
Best Practices For Writing DAGs In Airflow 2
▶︎
Great Expectation Tutorial | Mastering Data Quality
▶︎
Open-Source Spotlight - Great Expectations (Data Quality Platform) - James Campbell
▶︎
Learn Practical Techniques for Applying Data Quality in the Lakehouse with Databricks
▶︎
Lakehouse data validation with Great Expectations in Microsoft Fabric
▶︎
Building a robust data pipeline with the dAG stack dbt, Airflow, Great Expectations
▶︎
RL for Agents Workshop - Deep Dive on Training Agents with RL and Open Source
▶︎
Marko Justinek - Consumer Driven Contract Testing using PactSwift
▶︎
Airflow 101: Essential Tips For Beginners
▶︎
Deep dive in to the Airflow scheduler
▶︎
Learn Snowflake in 2 Hours| High Paying Skills | Step by Step For Beginners
▶︎
Data Quality With or Without Apache Spark and Its Ecosystem
▶︎
Machine Learning in Production with Airflow
▶︎
Scheduling in Airflow
▶︎
Intro to Airflow for ETL With Snowflake
▶︎
Kafka System Design Deep Dive w/ a Ex-Meta Staff Engineer
▶︎
Scalable Data Ingestion Architecture Using Airflow and Spark | Komodo Health
▶︎