Building a Web Crawler from scratch in Zig

reference code: https://gist.github.com/nubskr/26cd0c... zig async IO video:    • A Practical Guide to Async IO in Zig   In this video, I build a web crawler in Zig using async I/O. It starts with one URL, fetches the page, extracts links, resolves relative URLs, filters out junk, and keeps crawling. Along the way, we build the core pieces of a crawler: request handling, URL queues, metadata tracking, link extractions, etc. Basically: Googlebot crawls billions of pages. This one is Zig, so we’ll see what happens. Chapters: 00:00 - Teaser 00:10 - Intro 01:05 - Building the CLI 03:25 - Fetching Webpages 08:08 - Indexing Webpages 14:25 - Resolving URLs 20:19 - One Thread Is Too Slow 21:04 - Multithreaded Web Scraping 32:47 - Building the Scraping Worker 37:56 - Hunting Bugs and Testing #zig #systemsprogramming #asynchronousprogramming