Submitted:
20 November 2025
Posted:
27 November 2025
You are already at the latest version
Abstract
This study presents an analytical evaluation of autonomous computational agents designed to solve complex, logicintensive challenges in dynamic digital environments. The framework explores how reasoning-based algorithms, adaptive learning mechanisms, and multi-step decision strategies can be employed to achieve goal-oriented automation without domainspecific pretraining. Through systematic experimentation, the research examines agentic behavior under variable task conditions, emphasizing adaptability, robustness, and problem decomposition efficiency. Empirical analysis demonstrates that structured reasoning and iterative self-correction can enhance performance consistency across diverse computational tasks. The findings contribute to advancing intelligent agent architectures in computer science, offering insights into scalable, general-purpose systems for autonomous decision-making and digital security problem solving.
Keywords:
1. Introduction
2. Related Work
3. Methodology
3.1. Experimental Framework
3.2. Agent Architectures

3.3. Evaluation Metrics
4. Experimental Design
4.1. Model Selection and Configuration
4.2. Environmental Enhancements
| Category | Baseline | Plan&Solve | ReAct | ReAct&Plan |
|---|---|---|---|---|
| General Skills | 60% | 85% | 97% | 100% |
| Reverse Engineering | 26% | 42% | 88% | 96% |
| Cryptography | 21% | 36% | 79% | 93% |
| Forensics | 46% | 58% | 83% | 91% |
| Binary Exploitation | 0% | 25% | 75% | 100% |
| Web Exploitation | 100% | 100% | 100% | 100% |
4.3. Implementation Details

5. Results and Analysis
5.1. Overall Performance
5.2. Category-Specific Analysis
5.3. Multiple Attempt Analysis
6. Ablation Study and Component Analysis
| Component | Modification | Success Rate | Change |
|---|---|---|---|
| Full Configuration | None | 95% | - |
| Reasoning Model | GPT-4o-mini instead of GPT-4o | 83% | -12% |
| Command Timeout | 10 seconds instead of 5 min- utes | 91% | -4% |
| Toolset | No additio nal packages |
91% | -4% |
| Planning Model | GPT-4o instead of o1-preview | 92% | -3% |
| Turn Limit | 10 instead of 30 | 90% | -5% |
| Planning Component | ReAct only, no planning |
91% | -4% |
| Structured Output | Standard command parsing |
85% | -10% |
7. DISCUSSION AND IMPLICATIONS
7.1. Architectural Insights
7.2. Practical Considerations
7.3. Limitations and Future Work
8. Conclusions
References
- Abramovich, T.; et al. (2024). “EnIGMA: Enhanced Interactive Generative Model Agent for CTF Challenges. arXiv:2409.16165.
- Aschenbrenner, L. (2024). “Situational Awareness: The Decade Ahead.” Situational Awareness AI.
- Bhatt, M. et al. (2024). “CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models.” arXiv:2404.13161. [4] OpenAI (2024). “Preparedness Framework.” OpenAI Safety Systems. [5] Phuong, M. et al. (2024). “Evaluating Frontier Models for Dangerous Capabilities.” arXiv:2403.13793.
- Project Zero (2024). “Project Naptime: Evaluating Offensive Security Capabilities of Large Language Models.” Google Security Blog.
- The White House (2023). “Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence.” White House Press Release. [8] UK Government (2023). “The Bletchley Declaration.” AI Safety Summit Proceedings.
- Yang, J.; et al. (2023). “InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback. arXiv:2306.14898.
- Yang, J. et al. (2023). “Language Agents as Hackers: Evaluating.
- Cybersecurity Skills with Capture the Flag.” NeurIPS Multi-Agent Security Workshop.
- Yao, S.; et al. (2023). “ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629.
- Yao, S.; et al. (2023). “Tree of Thoughts: Deliberate Problem Solving with Large Language Models. arXiv:2305.10601.
- Zhang, A. K.; et al. (2024). “Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models. arXiv:2408.08926.
- Shao, M.; et al. (2024). “NYU CTF Dataset: A Scalable OpenSource Benchmark for Evaluating LLMs in Offensive Security. arXiv:2406.05590.
- Amirin, A.; et al. (2024). “Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities. arXiv:2410.09114.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).