DuckLake Deep Dive: Build a Full Lakehouse with Just Parquet Files and DuckDB

DuckLake turns plain Parquet files into a real transactional lakehouse. No heavy setup, no Spark cluster. Substack article: https://open.substack.com/pub/thefull... DuckLake Docs: https://ducklake.select/docs/stable/ In this walkthrough, I’ll show you how to: Install DuckLake Read and write data Add and drop columns Run ACID transactions Partition data for performance By the end, you’ll have a lightweight, production-ready lakehouse running locally. 💡 Perfect for data engineers and analysts exploring next-gen analytics tools. 00:00 - Intro 00:40 - What problem does DuckDB solve 2:24 - Install DuckDB and DuckLake 4:23 - Create a DuckLake Table 6:15 - Read a DuckLake Table 8:39 - Read and update column values 12:41 - Look at DuckLake Snapshots 17:37 - Create and Drop new column for a table 20:38 - ACID transactions and commit messages 24:41 - Partitioning a DuckLake 28:49 - My final thoughts Hit subscribe if you want more raw first looks and real reactions to new data tools. You can also find me here: Substack: https://substack.com/@hoytemerson LinkedIn:   / hoytemerson   X: https://x.com/HoytEmerson