DuckDB & dataviz | End-To-End Data Engineering Project (3/3)

In this part 3 of the project, @mehdio explores how to build a Dashboard with Evidence using MotherDuck/DuckDb as a data source. You can visit the final dashboard here : https://www.duckdbstats.com/ 🎥 Part 1 :    • DuckDB & Python | End-To-End Data Engineer...   🎥 Part 2 :    • DuckDB & dbt | End-To-End Data Engineering...   ☁️🦆 Start using DuckDB in the Cloud for FREE with MotherDuck : https://hubs.la/Q02QnFR40 📓 Resources Github Repo of the tutorial : https://github.com/mehd-io/pypi-duck-... Template of evidence-MotherDuck : https://github.com/evidence-dev/evide... MotherDuck BI tools integration documentation : https://motherduck.com/docs/integrati... Evidence : https://evidence.dev/ ➡️ Follow Us LinkedIn:   / motherduck   X/Twitter :   / motherduck   Blog: https://motherduck.com/blog/ 0:00 Intro 0:53 Series & Architecture recap 2:05 Understanding how BI tools work 2:48 MotherDuck & Shared databases 4:08 Evidence introduction 5:18 Coding - exploring the template 12:58 Coding - building the pypi dashboard 21:05 Wrapping up #duckdb #dataengineering #sql #python #dataviz -------------------------------------- Learn how to track open-source project usage by building a data dashboard for PyPI package download stats. This DuckDB tutorial shows you how to gain valuable insights into your Python package's adoption, including download counts and user environments. We'll construct this entire data engineering project using MotherDuck as our serverless analytics platform and Evidence for a modern 'BI as code' approach to dashboarding. Discover MotherDuck, the serverless data warehouse powered by DuckDB, designed for fast analytical queries that are perfect for BI tools. We explain how to connect the Evidence BI framework to MotherDuck using a simple token. You'll learn how to access our pre-prepared PyPI stats dataset directly through a MotherDuck Share, eliminating complex data setup and letting you query the data immediately from any DuckDB client. Get a hands-on introduction to Evidence, an open-source framework that lets you build a powerful SQL dashboard using only Markdown and SQL. We'll guide you through setting up the project from a template in VS Code, configuring the MotherDuck data source, and understanding the core concepts of "BI as code," which brings software engineering best practices like version control to your analytics work. This guide demonstrates the rapid, iterative workflow of building your dashboard. See how to define a source SQL query to fetch data from MotherDuck and then use simple Evidence components in Markdown to create visualizations like line charts and bar charts. You'll learn how to adapt queries on the fly and see your dashboard update in real-time, making it easy to visualize PyPI download data for the DuckDB Python package. Master DuckDB optimization techniques for faster dashboard performance. We'll explore why data aggregation is crucial and show you how to write an efficient source query to transform millions of raw download records into an aggregated weekly view. This practical step reduces the data transferred to the client, a key consideration for any performant analytical dashboard and a common task in data engineering projects. Finally, we'll cover customizing your dashboard's layout and deploying your static JavaScript application to services like Vercel, Netlify, or Evidence Cloud. This video wraps up our end-to-end data project, showcasing a complete workflow from data ingestion to a production-ready BI dashboard using DuckDB, MotherDuck, and Evidence. Let us know what data observability or pipeline orchestration topics you'd like to see next Watch with full transcript & resources: https://motherduck.com/videos/duckdb-...