Preprint
Article

This version is not peer-reviewed.

Chain-of-Agents Planner: Multi-Agent LLM Collaboration for Long-Horizon Decision Making

  † Yu Yang and Mengyuan Ren contributed equally to this work.

Submitted:

29 October 2026

Posted:

30 October 2026

You are already at the latest version

Abstract
Large Language Models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks through multi-agent collaboration. However, existing approaches primarily focus on open-ended reasoning or simple task decomposition and lack structured frameworks for long-horizon decisionmaking with explicit planning, execution, and supervision. To address this, we propose Chain-of-Agents Planner (CoA-Planner), a framework that explicitly coordinates Planning Agents, Worker Agents, and a Supervisor Agent through a structured task graph. The Planning Agent decomposes complex tasks into a directed acyclic graph (DAG) of subtasks with dependencies,Worker Agents execute specialized subtasks, and the Supervisor Agent monitors progress and resolves conflicts at critical milestones. Our framework introduces adaptive role reconfiguration that dynamically adjusts agent assignments based on historical performance metrics. We evaluate CoA-Planner on several long-context benchmarks including document analysis (50K-120K tokens), codebase review, and technical report generation. Experimental results show that CoA-Planner achieves an average success rate of 67.9%, outperforming single-agent LLMs by 23.5 percentage points (pp), Chain-of-Thought with RAG by 17.9pp, and existing Chain-of-Agents frameworks by 10.4pp. The framework demonstrates efficient scaling with 4-5 worker agents and reduces error rates across all categories, particularly in context coverage and reasoning consistency.
Keywords: 
;  ;  ;  ;  

1. Introduction

Large Language Models (LLMs) have revolutionized artificial intelligence by demonstrating unprecedented capabilities in natural language understanding, reasoning, and generation [1]. Recent advances in multi-agent LLM systems have shown that collaborative approaches can significantly enhance performance on complex tasks [2,3]. However, current multi-agent frameworks face several critical limitations when dealing with long-horizon decision-making tasks.
First, existing approaches often lack explicit separation between planning, execution, and supervision roles. While frameworks like MetaGPT [3] and CAMEL [4] demonstrate effective agent coordination, they typically employ predefined workflows or peer-to-peer communication patterns that may not adapt well to dynamic task requirements. Second, most frameworks do not maintain structured representations of task dependencies, making it difficult to track progress, identify bottlenecks, or handle partial failures gracefully. Third, the absence of systematic agent performance monitoring prevents dynamic optimization of agent assignments based on demonstrated capabilities.
These limitations become particularly pronounced in long-context scenarios where tasks require processing documents exceeding 50K-200K tokens, such as comprehensive technical report generation, multi-document legal analysis, or large codebase review. Recent benchmarks like LongBench [5] and Loong [6] have revealed that even state-of-the-art LLMs struggle with long-context understanding, typically achieving success rates below 60% on realistic tasks.
To address these challenges, we propose Chain-of-Agents Planner (CoA-Planner), a structured multi-agent framework designed for long-horizon decision-making. Our key contributions are:
  • Explicit Task Graph Architecture: We introduce a DAG-based task representation that explicitly captures subtask dependencies, milestones, and execution constraints, enabling transparent progress tracking and failure recovery.
  • Role-Specialized Agent Design: We define three agent categories with clear responsibilities: Planning Agents for task decomposition, Worker Agents for specialized execution, and Supervisor Agents for monitoring and conflict resolution.
  • Adaptive Role Reconfiguration: We implement a performance-based mechanism that dynamically adjusts agent-to-task assignments using historical error rates and completion times, improving system efficiency over repeated executions.
  • Comprehensive Long-Context Evaluation: We evaluate CoA-Planner on diverse long-context benchmarks spanning document analysis, code understanding, and technical writing tasks with contexts ranging from 32K to 128K tokens. Although our framework is designed to handle contexts up to 200K tokens, we focus our empirical validation on the 32K-128K range to ensure thorough evaluation.
Experimental results demonstrate that CoA-Planner achieves substantial improvements over baseline methods. On document analysis tasks with 50K tokens, our framework attains 72.5% success rate compared to 52.3% for single-agent approaches. The framework exhibits efficient scaling properties, with optimal performance at 4-5 worker agents before communication overhead dominates. Error analysis reveals that CoA-Planner particularly excels at maintaining context coverage (average 76.0% vs. 51.5% for baselines) and reasoning consistency across long documents.

3. Methodology

3.1. CoA-Planner Architecture

The CoA-Planner framework consists of four main components as illustrated in Figure 1: Planning Agent, Task Graph, Worker Agents, and Supervisor Agent. Each component plays a distinct role in the collaborative decision-making process.
Planning Agent: Upon receiving a high-level task description T , the Planning Agent performs hierarchical decomposition to generate a set of subtasks S = { s 1 , s 2 , . . . , s n } . For each subtask s i , the agent determines:
s i = 〈 id i , desc i , input i , output i , deps i , role i 〉
where id i is a unique identifier, desc i is a natural language description, input i and output i specify data requirements, deps i ⊆ S defines prerequisite subtasks, and role i indicates the required agent specialization.
The Planning Agent constructs these dependencies as a directed acyclic graph (DAG) to ensure valid execution ordering. The planning process uses few-shot prompting with domain-specific examples to generate well-structured decompositions.
Task Graph: The Task Graph G = ( S , E ) explicitly represents the execution plan, where edges E = { ( s i , s j ) | s i ∈ deps j } encode dependencies. Each subtask node tracks execution state: pending, in_progress, completed, or failed. The graph supports:
  • Topological Execution: Tasks are executed following a topological order over the DAG, ensuring dependencies are satisfied.
  • Parallel Execution: Independent subtasks can execute concurrently on available Worker Agents.
  • Milestone Tracking: Critical nodes are designated as milestones where Supervisor review is required before proceeding.
Worker Agents: A pool of Worker Agents W = { w 1 , . . . , w m } executes subtasks according to their assigned roles. Each agent w j maintains:
w j = 〈 role , capabilities , performance history 〉
Agent capabilities are defined through role-specific prompts that configure the LLM for particular tasks (e.g., code analysis, document summarization, data extraction). The performance history tracks metrics including success rate, average completion time, and error patterns across previous executions.
When a subtask becomes available, the dispatcher selects an appropriate Worker Agent based on role matching and current load. The agent processes the task using the designated LLM with role-specific system prompts and executes any required tool calls (e.g., code execution, database queries, web search).
Supervisor Agent: The Supervisor Agent provides global oversight with two primary functions:
Progress Monitoring: The Supervisor continuously tracks task graph state, identifying bottlenecks and potential failures. At designated milestones, it reviews completed work using verification prompts tailored to the task domain.
Conflict Resolution: When Worker Agents produce conflicting outputs or errors occur, the Supervisor analyzes the situation and determines corrective actions, which may include task retry with modified parameters, reassignment to different agents, or dynamic plan revision in consultation with the Planning Agent.

3.2. Adaptive Role Reconfiguration

A key innovation in CoA-Planner is the ability to adaptively optimize agent-to-role assignments based on observed performance. Let P r , w denote the performance score of worker w on role r, computed as:
P r , w = α · SR r , w − β · ET r , w ET avg − γ · ER r , w
where SR r , w is the success rate, ET r , w is the average execution time normalized by the global average ET avg , and ER r , w is the error rate. Hyperparameters α , β , γ control the relative importance of each metric (set to 0.5, 0.3, 0.2 in our experiments).
After completing k tasks, the system performs role reconfiguration if the performance variance exceeds a threshold τ :
Var ( P r , W ) = 1 | W | ∑ w ∈ W ( P r , w − P ¯ r ) 2 > τ
The reconfiguration process reassigns roles to maximize the total expected performance:
max π : R → W ∑ r ∈ R P r , π ( r )
subject to workload balancing constraints that prevent overloading individual agents.
This adaptive mechanism operates entirely during inference, requiring no additional training. The system accumulates performance statistics during a brief warm-up phase (typically the first 3-5 tasks) before activating reconfiguration.

3.3. System Workflow

The complete execution workflow proceeds as follows:
  • Task Reception: User provides high-level task T with associated context (documents, code, data).
  • Planning Phase: Planning Agent decomposes T into task graph G using few-shot prompting with domain examples.
  • Execution Phase:
    • System identifies ready tasks (all dependencies satisfied)
    • Dispatcher assigns tasks to Worker Agents based on role requirements and availability
    • Worker Agents execute tasks in parallel where possible
    • Completed outputs are stored in shared context
  • Supervision Checkpoints: At milestones, Supervisor reviews intermediate results. If issues are detected, appropriate corrective actions are triggered.
  • Result Synthesis: Upon completing all tasks, results are aggregated and formatted as final output.
  • Performance Update: System updates agent performance histories and triggers reconfiguration if necessary.
This workflow ensures systematic progress through complex tasks while maintaining flexibility to adapt to dynamic conditions and optimize resource allocation.

4. Experimental Setup

4.1. Datasets and Tasks

We evaluate CoA-Planner on four categories of long-context tasks:
Document Analysis: We construct a dataset of 150 technical documents (academic papers, technical reports, policy documents) ranging from 50K to 120K tokens. Tasks include comprehensive summarization, key finding extraction, methodology analysis, and cross-document comparison. Ground truth is established through expert annotation.
Code Review: Using open-source repositories from GitHub, we select 100 codebases with 60K-100K tokens (multiple files). Tasks include architecture analysis, bug detection, code quality assessment, and feature documentation. We use established repositories with documented reviews as ground truth.
Legal Document QA: From the Loong benchmark [6], we extract 80 multi-document legal cases requiring information integration across 80K-150K token contexts. Tasks involve evidence location, comparative analysis, and reasoning chains.
Technical Report Generation: We create 60 scenarios requiring synthesis of multiple information sources (papers, documentation, data analyses) into structured technical reports of 5K-10K tokens, with input contexts of 80K-120K tokens.
For each category, we split data into training (60%), validation (20%), and test (20%) sets. Training data is used only for few-shot prompt engineering, not for model fine-tuning.

4.2. Baseline Methods

We compare CoA-Planner against four baseline approaches:
Single Agent: Direct prompting of state-of-the-art LLMs (GPT-4, Claude-3, Llama-3-70B) with full context and chain-of-thought encouragement.
CoT + RAG: Chain-of-thought prompting combined with retrieval-augmented generation using vector database (top-k=5 chunks, 512 tokens each).
Existing CoA (MetaGPT): The MetaGPT framework [3] adapted for our task domains with appropriate role definitions.
Reflexion: The Reflexion framework [13] enabling iterative refinement through self-reflection.
All methods use the same base LLMs and are provided with identical input contexts to ensure fair comparison.

4.3. Evaluation Metrics

We assess performance using multiple metrics:
Success Rate: Percentage of tasks where the system produces output meeting minimum quality criteria (verified by GPT-4 acting as an automatic judge, with human validation on 20% of the samples). A task is successful if it achieves ≥ 80 % completeness and ≥ 70 % accuracy scores.
Context Coverage: Fraction of input documents/sections that receive substantive analysis in the output, measured using attention-based overlap scores.
Execution Time: Wall-clock time from task initiation to completion, including all LLM calls and inter-agent communication. For CoA-Planner, we utilize parallel API calls with a limitation of 4 concurrent threads, which enables genuine parallel execution for independent subtasks while managing resource constraints.
Human Evaluation: Expert raters (3 per task) evaluate a subset of outputs on 1-10 scales for completeness, accuracy, coherence, efficiency, and overall quality. Inter-rater agreement is measured using Fleiss’ kappa.
Error Analysis: We categorize failures into five types: context missing (incomplete coverage), reasoning errors (logical inconsistencies), coordination failures (agent communication breakdown, applicable only to multi-agent systems), timeouts, and other errors (including formatting issues and edge cases). Error rates are calculated as the percentage of total tasks affected by each error type. Since a single failed task may exhibit multiple error types, the sum of error rates may exceed the overall failure rate.

5. Results

5.1. Overall Performance

Figure 2 presents comprehensive performance comparison across methods. CoA-Planner achieves superior results across all task categories, with particularly strong gains on longer context tasks. On Document Analysis (50K tokens), CoA-Planner reaches 72.5% success rate, representing 38.6% relative improvement over single-agent approaches (52.3%). The gap is even more pronounced for more complex tasks: on Legal QA (120K tokens), CoA-Planner achieves 63.7% versus 38.2% for single agents (66.8% relative gain).
Compared to CoT+RAG, which achieves 50.0% average success rate across the four task categories, CoA-Planner provides 35.8% relative improvement. The benefits over existing CoA frameworks are also substantial and consistent (18.1% average improvement over MetaGPT’s 57.5%), demonstrating that structured planning and supervision provide measurable advantages even over sophisticated multi-agent baselines.
Execution time analysis (Figure 2b) reveals that CoA-Planner maintains efficiency advantages despite increased coordination overhead. At 128K context length, CoA-Planner completes tasks in 155 seconds on average, 48% faster than single-agent approaches (298 seconds). This efficiency gain is achieved through parallel execution of independent subtasks on multiple API threads, which more than compensates for the communication overhead introduced by agent coordination. The optimized agent assignments from adaptive reconfiguration further contribute to this efficiency.
Context coverage (Figure 2c) shows substantial improvements with CoA-Planner achieving 76.0% average coverage across task categories, compared to 51.5% for single agents. This indicates that the multi-agent approach with explicit planning more reliably processes all relevant information from long contexts.
Table 1 provides granular results for each task category. The consistent pattern shows CoA-Planner’s advantages across diverse task types and complexity levels. Averaged across all four categories, CoA-Planner achieves 67.9% success rate, 76.0% coverage, and 94.5 seconds average execution time. The performance gains are particularly notable on tasks requiring comprehensive understanding (Legal QA, Technical Report) where structured planning and systematic coverage are most beneficial.

5.2. Ablation Studies

Figure 3a demonstrates the contribution of each component to overall performance. Removing the task graph structure (w/o Task Graph) reduces average success rate from 67.9% to 61.3%, showing that explicit dependency tracking provides significant value. The task graph enables better parallelization and clearer failure attribution compared to implicit coordination.
Disabling adaptive role reconfiguration (w/o Role Reconfig) yields 65.2% success rate, a 2.7 percentage point decrease. This indicates that learning from historical performance provides measurable but not critical benefits. The effect is more pronounced over longer experimental runs as agents accumulate more performance data.
The Supervisor Agent contributes substantially, with its removal (w/o Supervisor) reducing performance to 60.8%. The monitoring and conflict resolution capabilities prevent cascading failures and ensure quality at critical junctures. Using only the Planning Agent without Worker Agent specialization (Planning Only) drops performance to 55.4%, confirming that task decomposition alone is insufficient without appropriate execution mechanisms.
Scalability analysis (Figure 3b) reveals an optimal range of 4-5 Worker Agents for our task distribution. Performance increases from 55.8% with 2 agents to 67.9% with 4 agents as parallel execution opportunities are better exploited. However, marginal gains diminish beyond 5 agents (68.7%), while communication overhead grows substantially (from 25 to 48 messages per task). This overhead stems from increased coordination requirements and result synchronization across more agents.

5.3. Error Analysis

Figure 4 presents a detailed breakdown of error types. CoA-Planner substantially reduces context missing errors (8% vs. 35% for single agents), directly attributable to the task graph’s systematic coverage enforcement. Each subtask in the graph explicitly specifies required input documents, ensuring comprehensive processing.
Reasoning errors decrease from 42% (single agent) to 12% (CoA-Planner) due to specialized Worker Agents and Supervisor verification at milestones. Notably, coordination failures are inherently absent (marked as N/A) for single-agent systems (Single Agent and CoT+RAG), as these approaches involve no inter-agent communication. In contrast, multi-agent systems exhibit coordination failures: MetaGPT shows 25% while CoA-Planner maintains only 7%, demonstrating that our explicit task graph and Supervisor coordination mechanisms provide more reliable multi-agent orchestration than implicit message-passing approaches.
The error rates represent the percentage of total tasks affected by each error type. Since multiple error types can occur in a single failed task (e.g., a task may exhibit both context missing and reasoning errors), the sum of error rates can exceed the overall failure rate. For instance, with Single Agent having a 55.6% overall failure rate (100% - 44.4% success rate), the individual error percentages (35% + 42% + 18% + 12% = 107%) indicate significant error overlap, where most failed tasks exhibit multiple error types simultaneously.

5.4. Adaptive Reconfiguration Analysis

Figure 5a illustrates the learning dynamics of adaptive reconfiguration. Starting from initial (suboptimal) role assignments at 55.8%, the system gradually improves performance to 67.9% over 10 task iterations. Static role assignment shows only marginal improvement (55.8% to 63.8%) from general experience accumulation. The adaptive approach converges to near-optimal performance by iteration 7, with diminishing returns afterward.
Role distribution changes (Figure 5b) reveal that the system allocates more resources to data retrieval (30% to 35%) and code analysis (25% to 30%) while reducing allocation to report writing (25% to 20%) and quality checking (20% to 15%). This reflects the actual bottlenecks in our task distribution, where information gathering and analysis are more challenging than synthesis and verification. The system discovers this through observing completion times and success rates across role types.

5.5. Human Evaluation

Expert raters (inter-rater agreement Fleiss’ κ = 0.67 ) evaluated 50 randomly sampled outputs per method across five criteria (Figure 6). CoA-Planner achieves 8.4 average rating compared to 6.5 for single agents, 7.0 for CoT+RAG, and 7.5 for MetaGPT. The largest improvements appear in completeness (8.3 vs. 6.2) and efficiency (8.2 vs. 6.3), aligning with quantitative coverage and time metrics.
Qualitative feedback indicates that raters particularly appreciated CoA-Planner’s systematic coverage of all document sections and logical organization of findings. Common criticisms of baselines included missing important details (single agent), redundant retrievals (CoT+RAG), and occasionally disjointed outputs (MetaGPT). CoA-Planner’s task graph structure appears to promote more coherent and complete outputs that better satisfy expert requirements.

6. Discussion

6.1. Key Findings

Our experiments demonstrate several important findings about multi-agent collaboration for long-context tasks:
Structured Planning Matters: The explicit task graph provides substantial benefits (10-15% success rate improvement) over implicit coordination. The ability to visualize dependencies, track progress, and identify bottlenecks appears crucial for managing complex long-horizon tasks.
Supervision is Critical: The Supervisor Agent’s monitoring and intervention capabilities prevent cascading failures and improve reliability. In our error analysis, we found that 73% of prevented failures occurred through early detection at milestone checkpoints.
Optimal Agent Count is Task-Dependent: Our scalability analysis suggests 4-5 agents provide optimal performance for our task distribution. This reflects the inherent parallelism in typical long-context tasks (information gathering, analysis, synthesis, verification). Different task structures may have different optimal configurations.
Adaptive Optimization Provides Gradual Gains: Role reconfiguration yields measurable improvements (2-3% success rate). The effect compounds over time as performance histories accumulate, making it particularly valuable for production systems processing many similar tasks.

6.2. Limitations

Several limitations of CoA-Planner warrant discussion:
Coordination Overhead: The multi-agent architecture introduces communication and synchronization overhead. While our parallel execution mitigates this through concurrent API calls, systems with very fast base LLMs or very short tasks might not benefit from the added complexity.
Prompt Engineering Complexity: Effective use of CoA-Planner requires careful prompt design for each agent role and domain. The Planning Agent particularly needs good examples to generate appropriate task decompositions. This represents a barrier to adoption for new domains.
Evaluation Limitations: Our primary success metric relies on GPT-4 as an automatic judge for evaluating task outputs. While we validate 20% of samples with human evaluation to calibrate the scoring, this automated approach may introduce self-correction bias where the model favors outputs with similar stylistic patterns. Future work should explore more robust evaluation frameworks that combine multiple assessment methods and potentially use independent human evaluation at larger scale.
Failure Mode Diversity: While our error rate is lower than baselines, the types of failures are more diverse (coordination failures, resource conflicts). Debugging multi-agent failures can be more challenging than single-agent issues, requiring examination of message logs and task graphs.
Cost Considerations: Multiple LLM calls increase inference costs. For our average task with 4 workers, total token consumption is approximately 2.1× that of single-agent approaches. While the improved success rate (67.9% vs. 44.4% average for single agents) provides favorable cost-effectiveness for high-stakes tasks where failures are expensive, the doubled cost represents a significant practical consideration. For production deployment, we recommend careful cost-benefit analysis: CoA-Planner is most appropriate for complex, high-value tasks (legal document analysis, critical code review, strategic planning) where output quality justifies the additional expense. For routine or low-stakes tasks, simpler single-agent approaches may be more economically viable.

6.3. Future Directions

Several promising directions for future work emerge:
Learned Task Decomposition: Our Planning Agent currently uses few-shot prompting. Training specialized models or fine-tuning for task decomposition could improve plan quality and reduce prompt engineering burden.
Dynamic Agent Specialization: Rather than predefined roles, agents could develop specializations through meta-learning across diverse tasks. This would enable more flexible adaptation to novel domains.
Hierarchical Planning: Extending the task graph to support hierarchical subtask decomposition would enable handling of even more complex tasks with deeper planning horizons.
Theoretical Analysis: Formal analysis of coordination overhead, optimal agent allocation, and convergence properties of adaptive reconfiguration would provide stronger foundations for system design.

7. Conclusions

We have presented Chain-of-Agents Planner (CoA-Planner), a structured multi-agent framework for long-horizon decision-making with explicit planning, execution, and supervision components. Through extensive experiments on long-context tasks spanning 32K-128K tokens, we demonstrate that CoA-Planner achieves 67.9% average success rate across four diverse task categories, providing substantial improvements over single-agent LLMs (23.5pp absolute gain), Chain-of-Thought with RAG (17.9pp gain), and existing multi-agent frameworks (10.4pp gain over MetaGPT).
The key innovations of explicit task graph representation, role-specialized agents, adaptive reconfiguration, and hierarchical supervision prove effective for managing complexity in long-context scenarios. Ablation studies confirm the individual contributions of each component, while scalability analysis identifies optimal configurations (4-5 worker agents) balancing parallelism against coordination overhead.
Our error analysis reveals that CoA-Planner particularly excels at maintaining comprehensive context coverage (76.0% vs. 51.5% baseline) and reducing reasoning errors (12% vs. 42%). Human evaluation confirms these quantitative findings, with experts rating CoA-Planner outputs substantially higher across completeness, accuracy, coherence, and efficiency dimensions.
While coordination overhead and prompt engineering complexity present implementation challenges, the substantial performance gains justify these costs for high-stakes long-context applications including document analysis, code review, and technical report generation. The framework’s modular design enables adaptation to diverse domains through appropriate role definitions and task decomposition strategies.
Future work will explore learned task decomposition, dynamic agent specialization, and theoretical foundations for multi-agent coordination. As LLMs continue to improve, structured collaborative frameworks like CoA-Planner will be increasingly essential for unlocking their full potential on complex real-world tasks requiring long-horizon planning and execution.

References

  1. Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q.V.; Zhou, D.; et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 2022, 35, 24824–24837. [CrossRef]
  2. Talebirad, Y.; Nadiri, A. Multi-agent collaboration: Harnessing the power of intelligent LLM agents. arXiv preprint 2023. arXiv:2306.03314.
  3. Hong, S.; Zheng, X.; Chen, J.; Cheng, Y.; Zhang, C.; Wang, Z.; Yau, S.K.S.; Lin, Z.; Zhou, L.; Ran, C.; et al. MetaGPT: Meta programming for a multi-agent collaborative framework. arXiv preprint arXiv:2308.00352 2023. arXiv:2308.00352.
  4. Li, G.; Hammoud, H.A.A.K.; Itani, H.; Khizbullin, D.; Ghanem, B. CAMEL: Communicative agents for "mind" exploration of large language model society. Advances in Neural Information Processing Systems 2023, 36.
  5. Bai, Y.; Lv, X.; Zhang, J.; Lyu, H.; Tang, J.; Huang, Z.; Du, Z.; Liu, X.; Zeng, A.; Hou, L.; et al. LongBench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024, pp. 3119–3137.
  6. Wang, M.; Chen, L.; Fu, C.; Liao, S.; Zhang, X.; Wu, B.; Yu, H.; Xu, N.; Zhang, L.; Luo, R.; et al. Leave no document behind: Benchmarking long-context LLMs with extended multi-doc QA. In Proceedings of the Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024, pp. 5627–5646.
  7. Wang, L.; Xu, W.; Lan, Y.; Hu, Z.; Lan, Y.; Lee, R.K.W.; Lim, E.P. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint 2023. arXiv:2305.04091.
  8. Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K.; Cao, Y. ReAct: Synergizing reasoning and acting in language models. In Proceedings of the International Conference on Learning Representations, 2023.
  9. Cao, H.; Ma, R.; Zhai, Y.; Shen, J. LLM-Collab: A framework for enhancing task planning via chain-of-thought and multi-agent collaboration. Applied Computing and Intelligence 2024, 4, 328–348.
  10. Du, Y.; Li, S.; Torralba, A.; Tenenbaum, J.B.; Mordatch, I. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint 2023. arXiv:2305.14325.
  11. Guo, T.; Chen, X.; Wang, Y.; Chang, R.; Pei, S.; Chawla, N.V.; Wiest, O.; Zhang, X. Large language model based multi-agents: A survey of progress and challenges. In Proceedings of the Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI), 2024, pp. 8048–8057.
  12. Liu, Z.; Zhang, Y.; Li, P.; Liu, Y.; Yang, D. Dynamic LLM-Agent Network: An LLM-agent Collaboration Framework with Agent Team Optimization. arXiv preprint 2023. arXiv:2310.02170.
  13. Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; Yao, S. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems 2023, 36, 8634–8652. [CrossRef]
Figure 1. Architecture of Chain-of-Agents Planner showing the interaction between Planning Agent, Task Graph, Worker Agents, and Supervisor Agent. Solid arrows indicate primary data flow, while dashed arrows represent monitoring signals.
Figure 1. Architecture of Chain-of-Agents Planner showing the interaction between Planning Agent, Task Graph, Worker Agents, and Supervisor Agent. Solid arrows indicate primary data flow, while dashed arrows represent monitoring signals.
Preprints 241120 g001
Figure 2. Performance comparison across different methods. (a) Success rates on four task categories with varying context lengths. (b) Average execution time as a function of context length. (c) Document coverage rates across different task categories. CoA-Planner consistently outperforms baselines across all metrics.
Figure 2. Performance comparison across different methods. (a) Success rates on four task categories with varying context lengths. (b) Average execution time as a function of context length. (c) Document coverage rates across different task categories. CoA-Planner consistently outperforms baselines across all metrics.
Preprints 241120 g002
Figure 3. Ablation study results. (a) Impact of removing individual components from CoA-Planner, showing average performance across all four task categories. (b) Scalability analysis showing success rate and communication overhead as functions of worker agent count. Optimal performance occurs with 4-5 agents.
Figure 3. Ablation study results. (a) Impact of removing individual components from CoA-Planner, showing average performance across all four task categories. (b) Scalability analysis showing success rate and communication overhead as functions of worker agent count. Optimal performance occurs with 4-5 agents.
Preprints 241120 g003
Figure 4. Error rate heatmap showing the distribution of different error types across methods. Error rates represent the percentage of total tasks affected by each error type; multiple errors can occur in a single failed task. N/A indicates error types that cannot occur in single-agent systems (e.g., coordination failures). CoA-Planner reduces all applicable error categories, with particularly strong improvements in context missing and reasoning errors.
Figure 4. Error rate heatmap showing the distribution of different error types across methods. Error rates represent the percentage of total tasks affected by each error type; multiple errors can occur in a single failed task. N/A indicates error types that cannot occur in single-agent systems (e.g., coordination failures). CoA-Planner reduces all applicable error categories, with particularly strong improvements in context missing and reasoning errors.
Preprints 241120 g004
Figure 5. Adaptive role reconfiguration analysis. (a) Learning curve showing performance improvement over iterations for adaptive vs. static role assignment. Performance stabilizes around iteration 7. (b) Evolution of role distribution as the system learns optimal agent-task assignments. Data-intensive roles (Retrieval, Analysis) receive more resources.
Figure 5. Adaptive role reconfiguration analysis. (a) Learning curve showing performance improvement over iterations for adaptive vs. static role assignment. Performance stabilizes around iteration 7. (b) Evolution of role distribution as the system learns optimal agent-task assignments. Data-intensive roles (Retrieval, Analysis) receive more resources.
Preprints 241120 g005
Figure 6. Human evaluation results across five criteria (1-10 scale). CoA-Planner receives consistently higher ratings than baselines, with strongest improvements in completeness and efficiency.
Figure 6. Human evaluation results across five criteria (1-10 scale). CoA-Planner receives consistently higher ratings than baselines, with strongest improvements in completeness and efficiency.
Preprints 241120 g006
Table 1. Detailed Performance Metrics by Task Category
Table 1. Detailed Performance Metrics by Task Category
Task Category Success Coverage Time (s)
Document Analysis (50K tokens)
Single Agent 52.3 58.7 142
CoT + RAG 58.6 65.2 135
MetaGPT 64.2 71.5 98
CoA-Planner 72.5 82.1 76
Code Review (80K tokens)
Single Agent 45.7 52.3 168
CoT + RAG 51.3 58.6 159
MetaGPT 58.9 66.8 118
CoA-Planner 68.3 75.9 92
Legal QA (120K tokens)
Single Agent 38.2 45.1 215
CoT + RAG 42.8 50.7 205
MetaGPT 51.3 59.2 145
CoA-Planner 63.7 71.3 112
Technical Report (100K tokens)
Single Agent 41.5 49.8 192
CoT + RAG 47.2 56.3 183
MetaGPT 55.8 64.7 128
CoA-Planner 67.1 74.5 98
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.