Reconciling Strict Latency SLOs with Low Cost in Cloud-based Services

Abstract: Cloud-based online services must contend with a load that exhibits significant fluctuations at sub-second granularity. Despite such load volatility, services must meet strict Service Level Objectives (SLOs), which forces a tradeoff between over-provisioning resources in order to meet SLO and achieving high resource utilization plus cost efficiency. None of the existing approaches, which include reactive and proactive autoscalers, serverless (FaaS), and hybrid clusters combining virtual machines (VMs) and serverless workers, are able to reconcile this tension under fine-grained load fluctuation. In this talk, I will introduce Spandana, a fresh take on hybrid clusters that successfully resolves this tension. The key to Spandana’s success lies in decoupling SLO enforcement from cost optimization via simple, dedicated mechanisms. I will detail Spandana and demonstrate its superiority to existing approaches in terms of SLO adherence, VM utilisation and cost efficiency. Bio: Boris Grot is a Professor in the School of Informatics at the University of Edinburgh, where he leads the EASE Lab. His research interests include server hardware and software stacks, networking, and datacenter-scale computing. Boris is a member of the MICRO and HPCA Halls of Fame and a recipient of multiple awards for his research. Boris was the Program Chair for MICRO 2022 and General Chair for HPCA 2024. He sits on the Board of Directors for ACM SIGARCH. Domain & Tag