Cache Me If You Can: Decentralize Your Distributed Caches With Hollow - Viswanathan Ranganathan

Join us at the premier vendor-neutral open source conference, where developers and technologists come together to collaborate, share knowledge, and explore the latest innovations and advancements in open source technology. Learn more at https://events.linuxfoundation.org/ Cache Me If You Can: Decentralize Your Distributed Caches With Hollow - Viswanathan Ranganathan, Independent Distributed caches are often used for scenarios that don't actually require them. For massive datasets (100's of GB's or more), distributed caches make sense—the data simply won't fit in a single node's memory. However, distributed caches tend to be overkill when working with smaller data sets (100s of MBs to 10s of GBs) that do fit in memory. Additionally, using traditional In-Memory caching libraries creates additional operational challenges, such as cache stampedes during TTL expiration, memory spikes during reloads, and long cold-start times that directly affect deployment velocity. This talk proposes an alternate, unconventional view: What if we could decentralize our cache while centralizing its preparation? We'll discuss how dataset distribution using Hollow (an open-source project by Netflix) enables applications to serve data from local memory with microsecond access latency while staying perfectly synchronized via delta-based updates. We'll cover: Design trade-offs that make this pattern ideal for GB-scale, read-heavy workloads. Delta-based updates that optimize cache reloads/refreshes. Zero-downtime updates applied in milliseconds without memory spikes.