ML for ML Compilers - Mangpo Phothilimthana | Stanford MLSys #80

Episode 80 of the Stanford MLSys Seminar Series! ML for ML Compilers Speaker: Mangpo Phothilimthana Abstract: Search-based techniques have been demonstrated effective in solving complex optimization problems that arise in domain-specific compilers for machine learning (ML). Unfortunately, deploying such techniques in production compilers is impeded by several limitations. In this talk, I will present an autotuner for production ML compilers that can tune both graph-level and subgraph-level optimizations at multiple compilation stages. We demonstrate how to incorporate machine learning techniques such as a learned cost model and various learning-based search strategies to reduce autotuning time. Our learned cost model has high accuracy and outperforms a heavily-optimized analytical performance model. In an evaluation across 150 ML training and inference models on Tensor Processing Units (TPUs), the autotuner offers up to 2.4x and an average 5% runtime speedup over the heavily-optimized XLA compiler. I will outline how we deploy the learning-based XLA autotuner at datacenter scale to automatically tune the most heavily-used production models in Google's fleet everyday. The deployed tile size autotuner has been saving approximately 2% of fleetwide TPU compute time. We recently released a public dataset (https://github.com/google-research-da...) for the learned cost model, and host an on-going Kaggle competition on the dataset (https://www.kaggle.com/competitions/p...) to promote more research in ML for Systems. Bio: Phitchaya Mangpo Phothilimthana is a Staff research scientist at Google DeepMind (previously Google Brain), where she leads Machine Learning for Machine Learning Compilers effort (one of Google Brain moonshots in 2020). Her research interests include compilers, machine learning for systems, program synthesis, and energy-aware computing. Mangpo received an undergraduate degree in Computer Science from MIT and PhD from UC Berkeley. Mangpo was a recipient of Microsoft Research PhD Fellowship and Qualcomm Innovation Fellowship. Slide deck: https://drive.google.com/file/d/18tzj... Dataset: github.com/google-research-datasets/tpu_graphs Competition: kaggle.com/competitions/predict-ai-model-runtime -- Stanford MLSys Seminar hosts: Simran Arora, Dan Fu Twitter:   / simran_s_arora     / realdanfu​   -- Check out our website for the schedule: http://mlsys.stanford.edu Join our mailing list to get weekly updates: https://groups.google.com/forum/#!for... #machinelearning #ai #artificialintelligence #systems #mlsys #computerscience #stanford