문샷 AI, Kimi K3, 48시간 만의 자율 반도체 설계, 독자적 소프트웨어 스택과 컴파일러 구축(26.07.18)

00:00 The Miracle of 48 Hours of Autonomous Semiconductor Design 02:53 3T-Class Open Model Kimi K3 06:21 Confronting Proprietary Closed Ecosystems 08:34 RTL Autonomous Design and Error Control Loops 11:40 Ultralight Inference Accelerator K3 Nano 13:47 Algorithm Fixed Accelerator Design 15:04 Million Context Windows and Short-Term Memory 17:05 Building Autonomous MiniTriton Compiler 17:54 Performance Comparison with the Official Triton Stack 19:42 Kimi Delta Attention Technology 20:18 Power Savings via Attention Residuals 22:10 Stable LatentMoE Resource Control 22:47 Creating 3D Game Environments from Text 24:35 Vertical Closed Loops and Evolving Machines Overview of Kimi K3 and Architectural Innovations Developed by the Chinese AI company Moonshot AI, Kimi K3 is the world's first 3T-Class open hybrid model with 2.8 trillion parameters. This is a Master of Experts (MoE) model. It supports a massive context window capable of processing up to 1,048,576 tokens and natively provides multimodal capabilities encompassing text, images, and videos. This model incorporates various innovative technologies, such as Kimi Delta Attention (KDA), which maximizes decoding speed by linearly controlling computational complexity, and the Attention Residuals structure, which solves the information dilution problem found in deep neural networks. Additionally, through Stable LatentMoE technology, it maximizes computational efficiency by selectively activating only 16 out of 896 distributed experts per turn. Autonomous Semiconductor Chip Design in Just 48 Hours The most groundbreaking achievement demonstrated by the Kimi K3 is the autonomous design of its own inference accelerator chip, the Kimi K3 Nano, in just 48 hours without any intervention from human designers. For design automation, it utilized a fully open-source Electronic Design Automation (EDA) toolchain and the Nangate 45nm library instead of commercial tools. Through this process, it independently performed the entire silicon physical design process, ranging from architecture planning and logic synthesis to layout, wiring, and timing analysis. The completed chip features 1.46 million standard cells, an operating frequency of 100 MHz, and 0.277 MB of SRAM within an area of ​​4 mm². Through an INT4-based MAC array with built-in fused dequantization logic, it achieved remarkable simulation performance, decoding over 8,700 tokens per second. *Building a Proprietary Software Stack and Compiler* In addition to its hardware design capabilities, the Kimi K3 has demonstrated the ability to independently develop the core software stack that runs on top of this system. It autonomously implemented MiniTriton, a lightweight, proprietary compiler that generates low-level machine code by mimicking NVIDIA's commercial acceleration language, Triton. This compiler demonstrated computational efficiency equivalent to or exceeding that of standard Triton under specific workloads and performed mechanical optimizations flawlessly without loss of precision in real-world model training environments. *The Future of Digital Content Creation and Game Development* The Kimi K3 delivers powerful performance not only in advanced inference and agent capabilities but also in the field of digital content creation. When producing a teaser video announcing the launch of the company's model, the model itself selected 56 original clips and produced a sophisticated result through frame synchronization and editing synchronized with the music's beat. Furthermore, the ability to transform images and videos into fully interactive gameplay experiences paves a new path for future game development. In the initial demo, the potential of multimodal inference was demonstrated by the excellent rendering of a character riding a horse against a rainy farm backdrop. *Industrial Significance and Future Prospects* Amidst difficulties in securing cutting-edge infrastructure due to U.S. semiconductor export controls, the Kimi K3 has created a breakthrough to overcome the limitations of raw computational power through algorithmic innovation and advanced hardware-software collaborative design. This demonstration, in which an AI agent independently designed custom edge-dedicated silicon despite the constraints of legacy processes and open-source environments, is regarded as a critical turning point that will establish a new paradigm of autonomous design automation within the dedicated semiconductor ecosystem.