Benchmarking Time Series Foundation Models with sktime

🔊 Recorded at PyCon DE & PyData 2025, April 23, 2025 https://2025.pycon.de/program/GUKTNX/ 🎓 sktime's benchmarking capabilities enable systematic evaluation of time series foundation models against traditional approaches for data-driven decision making. Speakers: Benedikt Heidrich Description: In this presentation, Dr. Benedikt Heidrich examines the benchmarking capabilities of sktime for evaluating time series foundation models. The talk explores how sktime, an open-source Python toolbox for time series machine learning, provides unified interfaces for comparing different forecasting approaches, including emerging foundation models like Morai, Tiny Times Mixer, and Kronos. Dr. Heidrich demonstrates sktime's benchmarking module through practical examples of energy demand forecasting and hierarchical data forecasting, highlighting the importance of robust evaluation across different datasets and scenarios. The presentation emphasizes that while foundation models show promise, their effectiveness varies significantly depending on the specific use case and data characteristics. The benchmarking framework enables reproducible, verifiable comparisons between traditional forecasting methods and foundation models through features such as cross-validation splitting, multiple evaluation metrics, and runtime analysis. Dr. Heidrich illustrates how sktime's benchmarking capabilities can help practitioners make informed decisions about model selection by providing detailed insights into model performance across different time horizons and data hierarchies. The key takeaway is that systematic benchmarking is essential for determining whether foundation models offer advantages over traditional approaches for specific time series applications. ⭐️ About PyCon DE & PyData: The PyCon DE & PyData conference unite the Python, AI, and data science communities, offering a unique platform for collaboration and innovation. The PyCon DE & PyData 2025 conference, provided an exceptional experience, fostering deeper connections within the Python community while showcasing advancements in AI and data science. Attendees enjoyed a diverse and engaging program, solidifying the event as a highlight for Python and AI enthusiasts nationwide. Follow us: • LinkedIn:   / 28908640   • X: https://www.x.com/pyconde Links: • Conference website: http://pycon.de • Other sessions: https://2025.pycon.de/talks/ The conference is organized by • Python Softwareverband e.V.: http://pysv.org • NumFOCUS Inc.: http://numfocus.org • Pioneers Hub gemeinnützige GmbH: http://pioneershub.org If you enjoyed this session, please like, comment, and subscribe to our channel for more insightful talks and discussions. Share this video with your network to spread the knowledge! Hashtags: #Python #PyConDE #PyData #OpenSource #AI #DataScience #MachineLearning #SoftwareDevelopment #LLMs #Community Acknowledgements: Special thanks to all the volunteers and sponsors who made this event possible. About: Python Softwareverband e.V.: PySV is a non-profit that promotes the use and development of Python in Germany through events, education, and advocacy, fostering an open Python community. NumFOCUS Inc. supports open-source scientific computing by providing financial and logistical support to key projects like NumPy and Jupyter, promoting sustainable development and collaboration. Pioneers Hub gemeinnützige GmbH: is a non-profit fostering innovation in AI and tech by connecting experts and promoting knowledge exchange through events and collaborative initiatives. www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.