TUM AI Lecture Series - Building generative world models: progress and challenges (Ruiqi Gao)

Abstract: Equipping AI models with the ability to imagine, reason, and act in the physical world is a crucial step toward achieving Artificial General Intelligence (AGI). Generative world models, which simulate our world with high fidelity, offer a promising avenue for training and evaluating these embodied AI agents. In this talk, I will present recent progress in this field. I will begin by covering the latest advancements in video generation models and how they can be controlled through various actions, marking a key milestone in developing large-scale world models. Next, I will discuss how these video models can be integrated to interact with other modalities, such as audio and text. Finally, I will address the critical limitations of current video-based world models, such as ensuring long-term consistency and interactivity, and propose potential solutions to overcome these challenges.