LabStore - Part 1 - Building an Object Store in Go: How Hard Can It Be?

Let's learn all about the inner workings of S3 object stores, along with enough Go to start building your own solution. As MinIO distances itself from Open Source Software, losing karma with the community, there's no better time to discover more on how to build our very own S3-compatible object store! So let's delve into the inner workings of the AWS S3 Object Store API, learning the basics of Go, and introducing LabStore, a new project by IllumiKnow Labs, where, together with the community, we'll build yet another OSS solution for object storage, this time ensuring that it remains open and free—no features removed! Embark in this series with IllumiKnow Labs, and let's see where this journey takes us. Hopefully you'll learn a lot along the way, and don't forget to comment below, if you have requests, suggestions, or feedback! 📺 Watch the full LabStore series here ►    • LabStore   ► In part 1, we present LabStore's backend as a Go project, and the S3 requests we prioritize. ► In part 2, we delve into SigV4, the authorization signature algorithm used by S3 to authenticate requests and individual chunks. ► In part 3, we test alternative open source solutions and setup a benchmark for these solutions, comparing them with LabStore. ► In part 4, we cover the implementation and integration of IAM (Identity and Access Management). ► In part 5, we delve into designing a command line interface to manage LabStore. ► In part 6, we explore TUI development in Go, building an S3 client and file explorer for the CLI. 🔗 Resources: 💻 GitHub repo: https://github.com/IllumiKnowLabs/lab... 📝 Blog post: https://datalabtechtv.com/posts/labst... 🐹 Go: https://go.dev/ 📚 AWS SigV4: https://docs.aws.amazon.com/AmazonS3/... 📚 AWS S3 API: https://docs.aws.amazon.com/AmazonS3/... 📚 AWS IAM API: https://docs.aws.amazon.com/IAM/lates... "Go Gopher" by Renée French is licensed under CC BY 4.0 🔔 Subscribe for more deep dives data science, data engineering, and all things data! ☁️ Follow the channel on Bluesky: https://bsky.app/profile/datalabtecht... 🎧 Join our community's Discord:   / discord   👽 Join our subreddit:   / datalabtechtv   💻 Check out the channel's GitHub: https://github.com/DataLabTechTV 🌐 Visit our website for blog posts: https://datalabtechtv.com #softwareengineering #s3 #golang #objectstorage #objectstore #aws 00:00:00 Intro 00:01:08 MinIO - End of a Chapter 00:05:58 AWS S3 API 00:09:37 Planning & Priorities 00:13:51 Why Go? 00:19:55 Go: Composition 00:21:38 Go: Identifier Visibility 00:21:54 Go: Serialization 00:23:24 Go: Enums 00:25:38 Go: Methods 00:26:35 Go: Error Handling 00:30:13 Go: Custom Errors 00:32:40 Go: Panic 00:34:45 Go: Conventions 00:37:19 Go: net/http 00:43:02 Introducing LabStore 00:52:37 GitHub Org: IllumiKnow Labs 00:54:10 LabStore Prototype Demo 00:58:01 Outro