Accelerating Queries at CoddSpeed: Hardware Coprocessors on the Road to Production

This presentation will reveal how Microsoft's Gray Systems Lab builds a universal acceleration substrate. Using the TQP GPU coprocessor as an example, it will share the engineering philosophy and practical guidance from over 60 engineers on implementing cutting-edge hardware technology in complex commercial systems, including lessons on avoiding pitfalls. Presentation Topic: Accelerating Queries at CoddSpeed: Hardware Coprocessors on the Road to Production Abstract: Modern cloud platforms are continuously launching a series of dedicated hardware—GPUs, FPGAs, DPUs, AI accelerators, etc.—which have the potential to deliver significant performance improvements for analytical query processing. Ideally, a universal coprocessor design could allow us to integrate this potential into mature commercial database engines. In practice, however, this is exceptionally difficult. The real challenges often lie not in the kernels themselves, but in the type system, ANSI semantics, data layout encoding, hardware availability, and the engineering process rigor required by a production codebase relied upon by tens of thousands of customers. This talk will focus on CoddSpeed, an initiative within Microsoft's Fabric data warehouse aimed at building a universal hardware-accelerated query execution infrastructure to support the integration of different coprocessors over time. We will outline the recurring issues such integrations must address, the architectural decisions we made, and the unexpected real-world production scenarios that inevitably emerge once moving beyond controlled workloads. Drawing from our experience with the TQP (Tensor Query Processor) GPU coprocessor, which evolved from a research prototype into a system built by over 60 engineers, we provide a candid case study on transitioning accelerator research from the lab to production.