How xAI Scales Image & Video Processing with Ray | Ray Summit 2025

At Ray Summit 2025, Zhibei Ma and Kai-Hsun Chen from xAI share how the company is building a high-performance data processing stack to power some of the world’s most advanced multimodal AI models. They explain why multimodal data is central to xAI’s mission and how meeting the extreme demands of large-scale training led them to develop a distributed data pipeline built on Ray Core and KubeRay. This system enables efficient processing of massive image and video datasets with linear scalability and robust fault tolerance in production environments. In this talk, they present the architecture of xAI’s Ray-based data pipeline and the strategies used to achieve high availability and operational simplicity at supercluster scale. If you’re working on multimodal AI, large-scale data pipelines, or distributed training infrastructure, this session offers deep technical insight from real-world deployment. Liked this video? Check out other Ray Summit breakout session recordings Subscribe to our YouTube channel to stay up-to-date on the future of AI!    / anyscale   🔗 Connect with us: LinkedIn:   / joinanyscale   X: https://x.com/anyscalecompute Website: https://www.anyscale.com/