Syeda Nahida Akter | Front-Loading Reasoning: Why Reasoning Data Belongs in Pretraining

How should reasoning data be used when training large language models? In this talk, Syeda Akter (Carnegie Mellon University and NVIDIA) shares research showing that introducing reasoning data during pretraining creates a lasting advantage that amplifies the benefits of post training. She also discusses reinforcement learning in pretraining, synthetic reasoning data, and new directions for building stronger reasoning models.