Lightweight Analytics at Scale: Hands-On with DuckDB & Smallpond in Python

In this workshop, you’ll learn how to build powerful yet lightweight data workflows using Python, DuckDB, and Smallpond. We’ll explore how DuckDB enables fast, in-process SQL analytics on massive datasets without heavy infrastructure, and how Smallpond extends those capabilities into distributed, collaborative, or cloud-friendly environments. Timeline: 00:00:45 Introduction PyLadies Amsterdam 00:03:59 Workshop general intro 00:06:48 Introduction Valery C. Briz 00:08:39 What is Smallpond? 00:11:34 Why Smallpond is interesting? 00:14:22 Smallpond backend: DuckDB 00:15:21 Smallpond backend: 3FS (Fire-Flyer File System) 00:18:04 Smallpond backend: Ray 00:20:10 Smallpond backend: MinIO 00:21:23 How Smallpond works 00:26:05 Infra for the workshop and workshop setup with step by step explanations 00:52:30 Smallpond practice locally with the live debugging incl testing pandas, DuckDB, Ray, MinIO, Smallpond 01:24:07 Smallpond practice in Google Colab with comparison to pandas functionality: read a parquet file, save a data frame as a parquet file, filter conditions, map, apply 01:27:36 Smallpond specifics: partial SQL query, repartitioning output 01:31:05 Announcements GitHub Repo https://github.com/pyladiesams/lightw... Speaker: Valery Calderón Briz   / valerybriz   Valery is a Pythonista, Senior Data Engineer, Founder of the Python Guatemala community, previously co-leader in PyLadies Madrid and PyLadies Mexico City.