맥북에서 초거대 4000억 AI 돌려봤습니다... 속도도 빠르다? Flash-MoE 분석 || 메모리는 더욱 중요해질겁니다

We covered the remarkable case of a massive 397B-class AI model running on a MacBook. While it may appear to be a simple demo on the surface, it is actually closer to a signal showing how AI inference structures are evolving. In this video, we systematically examine why Flash-MoE was possible and the differences between Dense and MoE models. In particular, we provided an easy-to-understand explanation of how attention, FFN, and expert layers are separated and selected within a single layer. We also discussed why it is possible to compute by reading only a portion from an SSD without loading the entire model into memory. In this process, we explored the possibility that SSDs could evolve from simple storage devices into layers supporting inference. We also discussed how, ultimately, what matters is not just GPU performance, but how HBM, DRAM, and SSDs are distributed. We summarized the point that future competition in AI semiconductors could expand beyond simple computational performance to a competition in memory tiering. Finally, we addressed the reasons why this structure cannot be immediately applied to all models, as well as its realistic limitations. This video focuses on understanding the big picture of where future AI infrastructure is headed through the example of Flash-MoE. Written by Error Edited by Jin-i Lee [email protected]