The Illusion of Multi-Agent Advantage（2606.13003）【論文解説シリーズ】

[A Compass for the AI ​​Era] Paper Commentary Series The Illusion of Multi-Agent Advantage Prathyusha Jwalapuram, Hehai Lin, Chuyuan Li, Fangkai Jiao, Sudong Wang, Yifei Ming, Zixuan Ke, Chengwei Qin, Giuseppe Carenini, Shafiq Joty https://arxiv.org/abs/2606.13003 ⭐️ Author Organizations and Abbreviations Salesforce Research HKUST Guangzhou (Hong Kong University of Science and Technology, Guangzhou Campus) University of British Columbia (UBC) Nanyang Technological University (NTU) ⭐️ Problem Solved The industry consensus that "multi-agent systems (MAS)" combining multiple LLM agents are superior to single-agent systems is based on research where the single-agent systems being compared are weak or where computational costs are not controlled. This creates a fundamental problem: it's impossible to determine whether "MAS is truly superior or simply using more resources." Furthermore, it was pointed out that the benchmarks used for evaluation were primarily static inference tasks, potentially failing to leverage MAS's inherent strengths of parallelization, context isolation, and role-sharing. This research addresses this issue by achieving the following: Conducting a rigorous cost-inclusive comparison with a robust SAS baseline called Chain-of-Thought Self-Consistency (CoT-SC) across four models and five benchmarks: GPT-4o, GPT-5, GPT-OSS, and Gemini-2.5-Pro. Creating a novel synthetic multi-hop financial inference (SMFR) benchmark (588 test samples) that incorporates all the conditions in which MAS excels (parallelization, context weighting, and subtask decomposition). Achieving a diagnostic design that distinguishes between the "potential of the MAS concept" and the "failures of automated generation design" by using a manually designed Expert-MAS as a control. Dissecting the architecture of six frameworks—DyLAN, MAS-Zero, AFlow, ADAS, MaAS, and MAS-Orchestra—and visualizing three patterns of functional breakdown (immediate synchronization, positional bias, and ensemble degeneration). ⭐️ Core of the Paper The core of this paper is not an assertion that "MAS is wrong," but rather a diagnosis that "current automated generation MAS architectures, despite their complexity, do not lead to functional division of roles and fail to demonstrate a cost-justifiable advantage over CoT-SC." The fact that Expert-MAS achieved 96.5% of CoT-SC's SMFR (57%) demonstrates the validity of the MAS concept and proves that the problem lies in the immaturity of the automated search paradigm. ⭐️Key Points 1. Major Findings: The common notion that multi-agent systems are superior to single-agent systems was rigorously verified for the first time using AI agent evaluation. MAS based on automated agent design failed to consistently demonstrate superiority in chain-of-sort self-consistency across multiple AI benchmarks, and a more than tenfold cost increase was confirmed in AI cost-effectiveness. On the other hand, the appropriately manually designed Expert-MAS improved the GPT-5 CoT-SC task from 57% to 96.5%, demonstrating the limitations of the automated design paradigm. 2. Methodology: Six frameworks, including DyLAN, MAS-Zero, and AFlow, were compared using a multi-agent approach, employing a rigorous AI benchmark design that included AI cost-effectiveness. The core of the diagnostic design was the use of SMFR, which was optimized for MAS, and Expert-MAS as control groups. Three failure patterns were identified through architectural analysis: role redundancy, positional bias, and ensemble design. Effective improvement measures include further verification of distributed MAS and diversification of exploration methods in automated agent design. 3. Research Limitations: This research is limited to centralized automated agent design, excluding distributed MAS. SMFR, introduced as an AI benchmark, is a single diagnostic task, and its generalizability to other domains has not been proven. Furthermore, the study is limited to diagnostic observation rather than causal proof of functional failure, and some models are constrained to single-run execution. Effective solutions include combining multi-agent comparisons across multiple domains, cross-sectional verification of Expert-MAS design principles, and causal intervention experiments. 4. Related Research: This study critically examines how previous multi-agent system research, such as DyLAN and MAS-Zero, ignored AI cost-effectiveness. Sharing the same concerns as related research that emphasized the importance of cost control, it newly categorizes three failure patterns specific to LLM agents: role redundancy, architectural bloat, and ensemble development. Its unique contribution lies in shifting the approach to AI agent evaluation from performance comparison to structural diagnosis. 5. Future Impact: This study encourages a shift from automated agent design to mechanistically verifiable role design. By establishing AI cost-effectiveness as a standard indicator, th...