EP 95. DeepSeek-V4 논문 읽기

On the morning of Sunday, April 26, 2026, we summarize the major news from the AI ​​industry over the past week and conduct an in-depth review of DeepSeek-V4. After covering key issues such as the launch of GPT-5.5 and Google Cloud Next, we delve deeply into how DeepSeek-V4’s model scaling (1.6T) and core changes—such as sparse attention, mHC, and the Muon optimizer—have transformed the structure of performance and costs (computation and KV cache). In particular, we examine from a technical perspective the design that significantly lowered costs in long-context training and inference, as well as the current state of the competitive landscape among Chinese frontier labs. 00:00:00 This Week's AI News Starts with GPT-5.5 and DeepSeek-V4 00:01:06 DeepSeek's Position in the Landscape of China's Frontier Labs 00:03:34 DeepSeek-V4 Model Expansion and Architectural Changes, Expanded to 1.6T 00:04:03 Sparse Attention Reducing Computational Burden and KV Cache 00:08:16 The Significance of Sparse Attention Through From-Scratch Training 00:12:10 Three Core Components Constituting Sparse Attention 00:17:10 Lightning Indexer Handling KV Cache and Top-K Selection 00:21:36 Non-differentiability of Top-K Selection and Training Instability 00:27:24 MLA Removal and Introduction of Muon Optimizer 00:30:57 DeepSeek-V4 Algorithm Organized Without En-Grams 00:31:18 MoE Pipeline That Boosted Training Infrastructure Optimization 00:34:08 Mega-kernel and FP4 Quantization for Infrastructure Efficiency 00:39:02 Pre-training Extended with 32T Tokens and Long-Context Learning 00:42:37 Anticipatory Routing for Training Instability 00:46:35 On-Policy Distillation and Rubric Reward for Refining Post-training 00:50:08 DeepSeek-V4 Benchmark Compared to Claude, GPT, and Gemini 00:54:31 Contributors, Huawei Chips, and the Backstory of Meta Muse Spark 00:56:39 Quick Summary of Cloud Next and GPT-5.5 News 01:00:00 Accelerated Development and Burnout as Seen Through Cat Wu Interview 01:02:08 The Competitive Landscape Shifting from Model Performance to Business Value 01:04:37 Wrapping Up Today's Discussion and Preview of the Next Episode Transcript: https://aifrontier.kr/ko/episodes/ep95