Preprints.org logo
Preprint
Article

This version is not peer-reviewed.

InternAgentHarness: A Scalable Synthetic Environment for Enhancing LLM Agentic Abilities

  † Equal contribution.

Submitted:

11 September 2026

Posted:

11 September 2026

You are already at the latest version

Abstract
Large language models (LLMs) are increasingly expected to act as generalist agents capable of solving complex real-world problems. Training such agents, however, requires stable and diverse environments that support repeated interaction with stateful, tool-augmented tasks and provide verifiable feedback. Despite recent progress, the development of robust LLM agents remains limited by the lack of realistic, scalable, and executable training environments. We present InternAgentHarness, a scalable synthetic environment for improving the agentic capabilities of LLMs. Built upon the InternBootcamp training framework [1], InternAgentHarness instantiates executable agent environments through a four-layer interface that unifies prompt generation, tool execution, interaction control, and reward computation. We further introduce \bootcampcli, an agent harness that automatically converts diverse agentic tasks into a Bootcamp-trainable paradigm. Unlike static benchmarks, InternAgentHarnessmakes evaluation actionable: observed failures can be systematically converted into new synthetic tasks, filtered trajectories, reinforcement learning rollouts, and subsequent re-evaluation under the same executable interface. We instantiate InternAgentHarness on a suite of 10 tasks covering both text-only and vision-based agent scenarios. Starting from Qwen3-VL-30B-A3B-Thinking, both supervised fine-tuning (SFT) and reinforcement learning (RL) on InternAgentHarness substantially improve performance over untuned model. These results suggest that InternAgentHarness provides a practical foundation for scalable synthetic agent environments and enables the continuous improvement of LLM agents through an iterative cycle of evaluation, synthesis, training, and refinement.
Keywords: 
LLM agent
;  
data synthetic
;  
post train
Subject: 
Computer Science and Mathematics  -   Artificial Intelligence and Machine Learning

1. Introduction

Large language models (LLMs) are increasingly expected to act as generalist agents rather than chatbots. A frontier agent must decompose high-level goals, invoke external tools, observe state changes, revise its plan, and terminate only when it can produce a verifiable answer. Such capabilities cannot be acquired from static prompts alone. They require repeated interaction with executable environments that expose stateful tasks, tool feedback, and reliable reward signals.
This requirement creates a practical bottleneck for agent development. On the one hand, many existing LLM-agent studies focus on benchmarks that provide realistic evaluation settings, including web navigation, code repair, scientific reasoning, and operating-system control [2,3,4,5,6]. These benchmarks are valuable because they expose where models fail under complex interactive conditions. However, most of them are not designed as scalable training environments. In particular, a failure case is often difficult to transform into a new task instance, a filtered trajectory, a reinforcement-learning rollout, or a reward-aligned re-evaluation protocol. As a result, model developers can more easily diagnose agent weaknesses than convert these weaknesses into the next round of training data. On the other hand, recent work on environment synthesis suggests a more scalable path for agent learning. A key requirement for general-purpose agents is access to diverse, realistic, and interactive environments [7]. Existing efforts mainly follow two directions. The first direction builds simulated environments, where LLMs serve as implicit textual world models that generate feedback for agent training [8,9,10]. Although such simulators are highly scalable, they are prone to hallucination and may deviate from real-world dynamics. The second direction constructs realistic environments by combining executable tools with grounded databases, thereby providing stronger support for complex interactions [11,12,13]. However, manually crafting such environments remains expensive and difficult to scale.
We present InternAgentHarness, a scalable synthetic environment for improving the agentic capabilities of LLMs. Built upon InternBootcamp [1], InternAgentHarness extends verifiable task scaling from static reasoning problems to executable agent environments. Each task is represented through a four-layer interface. Prompt generation defines task instances and model-facing instructions; tool execution exposes callable affordances and stateful backends; interaction control manages multi-turn feedback and termination; and reward computation verifies outcomes and provides training signals. This interface serves as the central abstraction that allows heterogeneous agent tasks to be sampled, served, solved, scored, logged, used for training, and evaluated under a unified environment contract.
To improve the scalability and automation of agentic environments, we introduce BootCampCLI, an agent harness that automatically converts diverse agentic tasks into a Bootcamp-trainable paradigm. BootCampCLI bridges the four-layer environment definition and the agent runtime. It loads agent, tool, and interaction configurations; constructs executable episodes; routes model actions through the environment; and records trajectories, tool feedback, and rewards in a reusable format. This harness is essential for scalable training: a task can be first validated as a single executable episode and then reused, without interface changes, for distributed trajectory collection, supervised fine-tuning (SFT) filtering, reinforcement learning (RL) rollouts, held-out evaluation, and failure-driven environment refinement.
This workflow makes evaluation actionable. Rather than ending with a leaderboard score, a failed run in InternAgentHarness produces structured evidence, including the prompt, model response, tool calls, interaction feedback, reward metadata, and final answer. These records can be filtered into supervised training data, replayed as on-policy rollouts, or inspected to improve task generation and reward design. As a result, the same environment supports an iterative cycle of evaluation, synthesis, training, and refinement.
We instantiate InternAgentHarness on a 10-task suite covering both text-only and vision-based agent scenarios. These tasks require scientific tool use, constrained optimization, financial reasoning, game interaction, long-horizon decision making, and visually grounded action selection. Starting from Qwen3-VL-30B-A3B-Thinking, both SFT and RL on InternAgentHarness substantially improve performance over untuned open baselines, with the largest gains observed on feedback-rich tasks. These results suggest that scalable synthetic environments can serve not only as evaluation platforms, but also as practical substrates for continuously improving LLM agents.
Our contributions are summarized as follows:
1.
We introduce InternAgentHarness, a scalable synthetic environment built on InternBootcamp that represents heterogeneous agent tasks through a unified four-layer interface covering prompt generation, tool execution, interaction control, and reward computation.
2.
We introduce BootCampCLI, an agent harness that converts diverse agentic tasks into a Bootcamp-trainable paradigm. BootCampCLI enables the same task interface to support local validation, distributed rollout, trajectory logging, reward-based training, supervised fine-tuning, reinforcement learning, re-evaluation, and environment refinement.
3.
We evaluate InternAgentHarness on a 10-task suite covering text-only and vision-based scenarios. Experiments show that environment-aligned SFT and RL substantially improve open-model agent performance, and that feedback-rich environments provide the strongest training signals.
The rest of the paper follows this structure. Section 2 discusses agentic post-training, benchmarks, and scalable environment synthesis. Section 3 presents the four-layer synthetic environment abstraction and BootCampCLI harness. Section 4 evaluates whether the environment supports heterogeneous task scale-up and effective post-training. Section 5 summarizes the implications and limitations.

3. Methods

Preprints 232722 g001
Figure 1. Representative tasks in InternAgentHarness. The task spans scientific design, optimization, games, finance, routing, and visual decision-making tasks. Each task is packaged as an executable agent environment with task-specific observations, tool affordances, feedback signals, and verifiable outcomes.
Figure 1. Representative tasks in InternAgentHarness. The task spans scientific design, optimization, games, finance, routing, and visual decision-making tasks. Each task is packaged as an executable agent environment with task-specific observations, tool affordances, feedback signals, and verifiable outcomes.
Preprints 232722 g001

3.1. Method Overview

InternAgentHarness is designed around the motivation stated in the introduction: agent improvement should not stop at measuring failures. A useful agent environment must make failures reusable, so that the same task can be generated, executed, scored, logged, converted into training data, optimized through reinforcement learning, and evaluated again. This requirement is difficult to satisfy with task-specific benchmarks, because each benchmark often defines its own prompt format, tool protocol, interaction rule, reward function, and logging convention. InternAgentHarness addresses this bottleneck by extending InternBootcamp [1] from verifiable task scaling to executable agent-environment scaling. The key abstraction is a Bootcamp-trainable episode. Each episode contains a hidden task identity, a model-facing prompt, optional tools or interaction backends, state transitions, reward computation, and trajectory records. By forcing heterogeneous tasks to compile into this common episode abstraction, InternAgentHarness makes task scale-up compatible with post-training and re-evaluation.
Preprints 232722 g002
Figure 2. System workflow of InternAgentHarness. A bootcamp specification defines instruction generation, tool schemas, interaction rules, state control, and reward computation. The same executable task interface supports trajectory generation, SFT warm-start, RL rollout, held-out evaluation, and failure-driven task refinement, forming a closed loop for continuous agent improvement.
Figure 2. System workflow of InternAgentHarness. A bootcamp specification defines instruction generation, tool schemas, interaction rules, state control, and reward computation. The same executable task interface supports trajectory generation, SFT warm-start, RL rollout, held-out evaluation, and failure-driven task refinement, forming a closed loop for continuous agent improvement.
Preprints 232722 g002
This design follows three principles. First, task generation and reward verification must remain grounded in hidden identity metadata, so that synthetic instances can be scaled without losing correctness. Second, tools and interactions must be executable rather than simulated only through model text, so that agents receive reliable environment feedback. Third, every rollout must produce reusable traces, because these traces are the bridge from evaluation to SFT, RL, and environment refinement.

3.2. Four-Layer Environment Interface

InternAgentHarness represents each agent environment through four layers: prompt, tool, interaction, and reward. The purpose of this separation is not only software modularity. It defines the minimum contract needed to turn a task into a scalable training environment.

Prompt Layer

The prompt layer defines how task instances are sampled and presented to the model. For each episode, an instruction generator samples an identity object containing hidden targets, constraints, instance parameters, and optional modality bindings. It then produces the model-facing instruction, including the task goal, available observations, action format, answer format, and constraints. The identity object is hidden from the model but remains accessible to tools, interaction handlers, and reward calculators. This separation allows InternAgentHarness to scale diverse task variants while keeping evaluation deterministic and verifiable.

Tool Layer

The tool layer exposes executable affordances through OpenAI-compatible function schemas. A tool specifies its name, description, typed arguments, return format, and lifecycle. During rollout, a model action is routed to the corresponding backend through create, execute, calc_reward, and release. This lifecycle binds the episode to isolated tool state, executes task-specific operations, returns observations or errors, optionally computes local metrics, and releases resources after completion. The same abstraction can wrap local functions, numerical solvers, databases, simulators, web services, or visual environments.

Interaction Layer

The interaction layer controls the multi-turn dynamics of an episode. Many agent tasks cannot be reduced to independent tool calls: the environment must reveal state-dependent observations, provide corrective feedback, enforce budgets, and decide whether the rollout should continue or terminate. InternAgentHarness therefore exposes an interaction interface that returns a termination flag, response content, turn-level score, and auxiliary metadata. This layer is crucial for feedback-rich training, because the model can revise its plan based on environment responses rather than producing a single static answer.

Reward Layer

The reward layer converts environment execution into a learning signal. Reward calculators extract the model’s final answer or intermediate outputs, compare them with the identity metadata, and return correctness or partial-credit scores. They can also encode format validity, constraint satisfaction, tool-use efficiency, budget adherence, trajectory quality, and multimodal grounding. Because reward computation is decoupled from model inference and backend execution, the same task can support rejection sampling, SFT filtering, RL rollout, held-out evaluation, and failure analysis.

3.3. BootCampCLI: Runtime Harness for Task Scale-Up

The four-layer interface defines what an environment contains; BootCampCLI defines how that environment is ingested by the agent runtime. Its role is to convert a heterogeneous task specification into a Bootcamp-trainable episode that can be reused across local validation, distributed rollout, training, and evaluation.

From Configuration to Executable Episode

BootCampCLI loads agent, tool, and interaction configurations; attaches tool schemas and tool instances; binds interaction handlers to the relevant agent nodes; and initializes the AgentRuntime. This process compiles the prompt, tool, interaction, and reward layers into a single executable episode. Task authors therefore only need to implement task-specific environment components and register them in configuration files, while the agent-side execution path remains shared across domains.

A Harness for Scale-Up

BootCampCLI is introduced as a scale-up interface rather than as a command-line convenience. It standardizes how prompts, model messages, tool calls, tool outputs, interaction feedback, rewards, and termination states flow through the harness. A task can first be debugged as one executable episode and then reused, without changing its interface, for distributed trajectory collection, SFT data filtering, RL rollouts, held-out evaluation, and failure-driven refinement. This is the mechanism that turns diverse agentic tasks into a Bootcamp-trainable paradigm.

Reusable Trajectory Records

During execution, BootCampCLI records the full episode trace, including prompts, model responses, tool calls, environment observations, interaction events, local rewards, final rewards, and final answers. These records are the artifact that makes evaluation actionable. Successful traces can be filtered into supervised trajectories, on-policy traces can be used by RL, and failed traces reveal whether the next iteration should change the prompt generator, tool affordances, interaction feedback, reward design, or instance distribution.

3.4. Scalable Rollout and State Isolation

Scaling synthetic agent environments requires many episodes to run concurrently while still preserving deterministic verification for each instance. To support this, InternAgentHarness separates the execution context into three complementary types of state. The task state contains instance-specific information, such as hidden identities, private constraints, target answers, and task parameters, which must remain isolated across episodes. The tool state maintains backend-specific runtime information, including simulator status, previous tool outputs, active resources, and visual observations, which may rely on shared infrastructure but should expose only episode-local views. The rollout state records the agent-facing interaction process, including conversation history, available tools, interaction status, rewards, and termination flags.
This design decouples what is private to a task, what is managed by shared tool backends, and what belongs to the agent’s current trajectory. As a result, many episodes can reuse expensive simulators or service backends without leaking hidden task information or corrupting each other’s interaction histories, thereby enabling scalable and deterministic parallel rollouts.

Distributed Tool Serving

Tools are served through a master-worker architecture. The master handles worker registration, routing, and load balancing, while workers host concrete tool implementations. A single-process mode supports local validation before scale-up. Because each rollout is associated with an instance ID, many episodes can share backend workers without mixing state. This design is important for tools such as scientific solvers, circuit simulators, game engines, financial calculators, and visual environments, where starting a new backend per rollout would be inefficient.

Structured Feedback and Errors

Executable environments naturally produce informative failures, including invalid arguments, infeasible states, timeouts, unsafe actions, and premature termination. InternAgentHarness surfaces these events as structured feedback instead of leaving them as opaque logs. The model can receive corrective interaction messages during the episode, while the trace retains enough metadata for later training and diagnosis. Thus, an error can become a negative example, a reward-shaping signal, or evidence that the environment specification should be refined.

3.5. Training and Evaluation Workflow

InternAgentHarness connects the components above into a continuous improvement loop:
1.
Task construction: sample hidden identities, modality bindings, constraints, and model-facing prompts.
2.
Environment registration: define tool schemas, backend logic, interaction rules, state transitions, and reward calculators.
3.
Runtime compilation: use BootCampCLI to convert the registered environment into an executable Bootcamp-trainable episode.
4.
Trajectory generation: collect model actions, tool calls, environment responses, interaction events, rewards, and final answers under a unified schema.
5.
Post-training: use high-quality traces for supervised warm-start and connect the same executable environments to RL rollouts.
6.
Re-evaluation: run held-out instances through the same interface and compute verifiable task-level scores.
7.
Refinement: inspect failures and update task generation, tool affordances, interaction feedback, reward computation, or instance distributions.
The workflow is intentionally trainer-agnostic. For SFT, InternAgentHarness exports conversation-style trajectories containing tool-call messages, tool outputs, interaction feedback, and final answers. For RL, the rollout side consumes the same prompts and tool schemas while receiving online environment feedback and rewards. For evaluation, the same task definition produces held-out instances and verifiable scores. In this way, InternAgentHarness makes evaluation, synthesis, training, and refinement part of one scalable environment loop.

4. Experiments

4.1. Experimental Setup

The experiments are designed to test the method’s central motivation: a synthetic agent environment should be useful not only for reporting scores, but also for scaling heterogeneous tasks and improving models through reusable interaction traces. We therefore evaluate three questions. First, can the same four-layer interface instantiate diverse text-only and vision-based agent tasks? Second, does environment-aligned post-training improve an open model that initially struggles with executable agent protocols? Third, what kinds of environments provide the strongest training signal for the next round of refinement?

Task Suite

We instantiate InternAgentHarness on 10 tasks using the same prompt, tool, interaction, and reward interface. The suite contains eight text-only tasks (battery, fenics, epidemic, mole, music, bot, finance, and escape) and two vision-based tasks (flangeplane and holdem). These tasks cover scientific tool use, constrained optimization, financial reasoning, game interaction, long-horizon decision making, and visually grounded action selection. The suite is intentionally heterogeneous: its purpose is to stress test whether InternAgentHarness can serve as a general executable environment substrate rather than a wrapper around one task family.

Environment Protocol

Each task is converted into a Bootcamp-trainable paradigm through BootCampCLI. The same task definition provides model-facing prompts, executable tool or interaction backends, state isolation, reward computation, and trace logging. For evaluation, we generate 100 held-out instances per task, yielding 1,000 evaluation episodes in total. All models are evaluated through the same deployed environment interface, and we report task-level success rates (%) together with separate averages for text-only and vision-based tasks.

Baselines

We compare against frontier proprietary systems and open-source models, including Gemini-3.1-Pro, Gemini-3-Pro [49], Kimi-K2.5 [50], Qwen3.5 [51], Qwen3-VL [52], Qwen3 [53], DeepSeek-V3.2 [54], and the Intern-S1 series [55]. This comparison helps separate three factors: general reasoning strength, multimodal perception, and the ability to operate inside an executable agent environment.

Training Configuration

Our trainable model is initialized from Qwen3-VL-30B-A3B-Thinking. We first perform supervised fine-tuning as a cold-start stage. Synthetic expert trajectories are generated by Kimi-K2.5 and Qwen3.5-397B-A17B-Thinking under the same environment interface and instruction templates used at evaluation time. We filter low-quality trajectories through rejection sampling with a score threshold of 0.8, producing 50k high-quality trajectories for SFT.
We then run reinforcement learning from the SFT checkpoint. For each task, InternAgentHarness generates 1k on-policy trajectories. We use GRPO with a learning rate of 1 × 10 − 6 , implemented with veRL [56]. The RL configuration uses 8 rollout trajectories, batch size 32, PPO mini-batch size 8, disabled KL loss, clipping bounds of 0.2 and 0.28, and 400 training steps on 32 NVIDIA H200 GPUs. Because all tasks share the same executable environment contract, the rollout pipeline can scale across heterogeneous tools while preserving comparable rewards and reusable traces.

4.2. Main Results

SFT Teaches the Model to Use the Environment Interface

As shown in Table 1, after SFT, Qwen3-VL-30B-A3B-Thinking improves from 3.5 to 31.8 on the text-only average and from 38.5 to 62.8 on the vision average. The improvement is not merely a gain in final-answer imitation. The SFT trajectories teach the model how to interpret task prompts, format tool calls, consume environment observations, respond to interaction feedback, and produce verifiable final outputs. This supports the role of BootCampCLI and the four-layer interface: once diverse tasks are represented in a shared episode format, supervised trajectories can align a general model to that executable environment protocol.

RL Further Improves Closed-Loop Agent Behavior

RL further improves the SFT checkpoint from 31.8 to 51.4 on the text-only average and from 62.8 to 73.9 on the vision average. The largest gains appear on tasks such as battery, fenics, bot, finance, and flangeplane. These improvements support the central claim of InternAgentHarness: when tasks expose executable feedback and verifiable rewards through a unified interface, post-training can optimize behavior inside the environment rather than only imitate expert traces.

Feedback-Rich Tasks Provide Stronger Training Signals

The gains are not uniform across tasks. Environments with informative intermediate feedback, such as battery, fenics, finance, and flangeplane, benefit most from SFT and RL. In contrast, sparse or brittle-feedback tasks such as epidemic and holdem remain challenging. This result gives a practical insight for scaling synthetic environments: adding more tasks is insufficient unless the interaction and reward layers expose signals that help the model improve its subsequent actions.

Evaluation Becomes a Source of Environment Refinement

The experiment also demonstrates why InternAgentHarness is more than a static benchmark. A low score identifies not only a weak model behavior, but also a possible environment-design bottleneck: the prompt may be under-specified, the tool affordance may be hard to discover, the feedback may be too sparse, or the reward may provide little guidance. Because InternAgentHarness records prompts, actions, tool calls, interaction events, rewards, and final answers under the same schema, these observations can directly drive the next cycle of synthetic task generation, trajectory filtering, RL rollout, and re-evaluation.

Takeaway

Overall, the 10-task suite supports the paper’s main motivation. InternAgentHarness scales heterogeneous agentic tasks through a common executable interface, BootCampCLI makes these tasks usable by the same training and evaluation harness, and environment-aligned SFT and RL substantially improve an open model initialized from Qwen3-VL-30B-A3B-Thinking. The remaining failures also clarify the next scaling direction: richer interaction feedback and more informative rewards are essential for turning synthetic task diversity into stronger agentic ability.

5. Conclusion

This paper presents InternAgentHarness, a scalable synthetic environment for improving the agentic capabilities of LLMs. The central motivation is that robust agents require more than stronger base models or static benchmarks. They require stable, diverse, and executable environments where models can repeatedly interact with stateful, tool-augmented tasks and receive verifiable feedback. InternAgentHarness addresses this need by extending InternBootcamp from verifiable task scaling to multi-turn agent environments. The core design is a four-layer environment interface that unifies prompt generation, tool execution, interaction control, and reward computation. On top of this interface, BootCampCLI converts heterogeneous agentic tasks into a Bootcamp-trainable paradigm, so the same task definition can support local validation, distributed rollout, trajectory logging, supervised fine-tuning, reinforcement learning, held-out evaluation, and failure-driven refinement. This makes evaluation actionable: failed episodes become structured evidence for new synthetic tasks, filtered trajectories, reward redesign, and re-evaluation rather than isolated leaderboard outcomes.
Our experiments on 10 text-only and vision-based tasks support this view. Starting from Qwen3-VL-30B-A3B-Thinking, both SFT and RL on InternAgentHarness substantially improve open-model agent performance. The results also reveal useful design signals: SFT mainly aligns the model to the executable environment interface, RL improves closed-loop behavior when rewards and feedback are informative, and feedback-rich tasks provide the strongest training signal. These findings suggest that scalable synthetic environments can serve as practical substrates for continuously improving LLM agents through an iterative cycle of evaluation, synthesis, training, and refinement.

6. Limitations and Future Work

InternAgentHarness is an initial step toward scalable synthetic agent environments. The current suite covers 10 tasks and focuses mainly on reasoning-intensive, tool-augmented, and visually grounded scenarios. Scaling to larger and more diverse task libraries is necessary to study cross-domain transfer and long-term environment evolution. Sparse-feedback tasks also remain challenging, indicating that future work should improve interaction design, reward granularity, and failure attribution. Finally, extending the same interface to richer digital environments, embodied agents, multi-agent collaboration, and human-in-the-loop refinement would further test the generality of the Bootcamp-trainable paradigm.

References

  1. Li, P.; Ye, J.; Chen, Y.; Ma, Y.; Yu, Z.; Chen, K.; Cui, G.; Li, H.; Chen, J.; Lyu, C.; et al. InternBootcamp Technical Report: Boosting LLM Reasoning with Verifiable Task Scaling. arXiv 2025, arXiv:2508.08636. [Google Scholar]
  2. Zhou, S.; Xu, F.F.; Zhu, H.; Zhou, X.; Lo, R.; Sridhar, A.; Cheng, X.; Bisk, Y.; Fried, D. WebArena: A Realistic Web Environment for Building Autonomous Agents. In Proceedings of the ICLR, 2024. [Google Scholar]
  3. Jimenez, C.E.; Yang, J.; Wettig, A.; Yao, S.; Pei, K.; Press, O.; Narasimhan, K. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? arXiv 2024. [Google Scholar]
  4. Li, X.; Chen, J.; Fang, X.; Ding, S.; Duan, H.; Liu, Q.; Chen, K. OPT-BENCH: Evaluating LLM Agent on Large-Scale Search Spaces Optimization Problems. arXiv 2025. [Google Scholar]
  5. Jansen, P.; Sheng, Z.; Khot, T.; Holtzman, A.; Choi, Y. ScienceWorld: Is Your Agent Smarter Than a 5th Grader? In Proceedings of the EMNLP, 2022. [Google Scholar]
  6. Xie, T.; Zhang, D.; Chen, J.; Li, X.; Zhao, S.; Cao, R.; Hua, T.J.; Cheng, Z.; Shin, D.; Lei, F.; et al. OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. arXiv 2024, arXiv:2404.07972. [Google Scholar]
  7. Tu, D.; Hao, H.; Yang, H.; Chen, Y.; Zhang, Y.K.; Xia, Z.; Yang, Y.; Sun, Y.; Liu, X.; Shen, F.; et al. ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training. arXiv 2026, arXiv:2602.06820. [Google Scholar]
  8. Feng, J.; Zhang, Y.; Zhang, C.; Lu, Y.; Liu, S.; Wang, M. Web World Models. arXiv 2025. [Google Scholar]
  9. Guo, J.; Yang, L.; Chen, P.; Xiao, Q.; Wang, Y.; Juan, X.; Qiu, J.; Shen, K.; Wang, M. GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators. arXiv 2025, arXiv:2512.19682. [Google Scholar]
  10. Li, Y.; Inan, H.A.; Yue, X.; Chen, W.N.; Wutschitz, L.; Kulkarni, J.; Poovendran, R.; Sim, R.; Rajmohan, S. Simulating environments with reasoning models for agent training. arXiv 2025, arXiv:2511.01824. [Google Scholar]
  11. Bai, H.; Taymanov, A.; Zhang, T.; Kumar, A.; Whitehead, S. WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks, 2026. arXiv.
  12. Cai, S.; Fang, R.; Wu, J.; Li, B.; Wang, X.; Jiang, Y.; Su, L.; Zhang, L.; Yin, W.; Zhang, Z.; et al. AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning. arXiv 2025. [Google Scholar]
  13. Wu, Y.; Peng, Y.; Chen, Y.; Ruan, J.; Zhuang, Z.; Yang, C.; Zhang, J.; Chen, M.; Tseng, Y.; Yu, Z.; et al. AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machines. arXiv 2026. [Google Scholar]
  14. DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. CoRR 2025. abs/2501.12948. [Google Scholar]
  15. Li, X.; Fang, X.; Ding, S.; Li, L.; Duan, H.; Liu, Q.; Chen, K. NP-Engine: Empowering Optimization Reasoning in Large Language Models with Verifiable Synthetic NP Problems. arXiv 2025. [Google Scholar]
  16. Li, X.; Li, Y.; Fang, X.; Ding, S.; Li, P.; Chen, Y.; Ma, Y.; Lyu, T.; Li, L.; Lin, D.; et al. Beyond Mode Collapse: Distribution Matching for Diverse Reasoning. arXiv 2026. [Google Scholar]
  17. Zhang, G.; Geng, H.; Yu, X.; Yin, Z.; Zhang, Z.; Tan, Z.; Zhou, H.; Li, Z.; Xue, X.; Li, Y.; et al. The Landscape of Agentic Reinforcement Learning for LLMs: A Survey. arXiv 2025, arXiv:2509.02547. [Google Scholar]
  18. Li, X.; Lyu, T.; Yang, Y.; Shan, L.; Yang, S.; Zhang, L.; Huang, Z.; Liu, Q.; Li, Y. Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning, 2026. arXiv.
  19. Ma, Y.; Li, L.; chen, Y.; Li, P.; Li, X.; Guo, Q.; Lin, D.; Chen, K. Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic, 2026. arXiv.
  20. Song, H.; Jiang, J.; Min, Y.; Chen, J.; Chen, Z.; Zhao, W.X.; Fang, L.; Wen, J. R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning. CoRR 2025. abs/2503.05592, [2503.05592]. [Google Scholar] [CrossRef]
  21. Jin, B.; Zeng, H.; Yue, Z.; Wang, D.; Zamani, H.; Han, J. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. CoRR 2025. abs/2503.09516. [Google Scholar]
  22. Dong, G.; Mao, H.; Ma, K.; Bao, L.; Chen, Y.; Wang, Z.; Chen, Z.; Du, J.; Wang, H.; Zhang, F.; et al. Agentic Reinforced Policy Optimization. CoRR 2025. abs/2507.19849, [2507.19849]. [Google Scholar] [CrossRef]
  23. Zhao, W.; Wang, X.; Ma, C.; Kong, L.; Yang, Z.; Tuo, M.; Shi, X.; Zhai, Y.; Cai, X. MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use. arXiv 2025. [Google Scholar]
  24. Li, P.; Li, L.; Sun, H.; Mai, W.; Chen, Y.; Li, X.; Shen, Y.; Ma, Y.; Sun, Y.; Cao, J.; et al. TL-GRPO: Turn-Level RL for Reasoning-Guided Iterative Optimization. arXiv 2026. [Google Scholar]
  25. Gao, J.; Fu, W.; Xie, M.; Xu, S.; He, C.; Mei, Z.; Zhu, B.; Wu, Y. Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL, 2025. arXiv.
  26. Li, K.; Zhang, Z.; Yin, H.; Zhang, L.; Ou, L.; Wu, J.; Yin, W.; Li, B.; Tao, Z.; Wang, X.; et al. WebSailor: Navigating Super-human Reasoning for Web Agent. CoRR 2025. abs/2507.02592, [2507.02592]. [Google Scholar] [CrossRef]
  27. Feng, L.; Xue, Z.; Liu, T.; An, B. Group-in-Group Policy Optimization for LLM Agent Training. arXiv 2025, arXiv:2505.10978. [Google Scholar]
  28. Li, Y.; Gu, Q.; Wen, Z.; Li, Z.; Xing, T.; Guo, S.; Zheng, T.; Zhou, X.; Qu, X.; Zhou, W.; et al. Treepo: Bridging the gap of policy optimization and efficacy and inference efficiency with heuristic tree-based modeling. arXiv 2025, arXiv:2508.17445. [Google Scholar]
  29. Li, X.; Lyu, T.; Li, Y.; Ma, Y.; Li, P.; Li, L.; Guo, Q.; Lin, D.; Chen, K. What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents, 2026. arXiv.
  30. Li, W.; Lin, J.; Jiang, Z.; Cao, J.; Liu, X.; Zhang, J.; Huang, Z.; Chen, Q.; Sun, W.; Wang, Q.; et al. Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RL. CoRR 2025. abs/2508.13167, [2508.13167]. [Google Scholar] [CrossRef]
  31. Yuan, H.; Xu, Z.; Tan, Z.; Yi, X.; Guang, M.; Long, K.; Hui, H.; Li, B.; Chen, X.; Zhao, B.; et al. MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs. arXiv 2025. [Google Scholar]
  32. Chen, Y.; Yan, L.; Sun, W.; Ma, X.; Zhang, Y.; Wang, S.; Yin, D.; Yang, Y.; Mao, J. Improving retrieval-augmented generation through multi-agent reinforcement learning. arXiv 2025, arXiv:2501.15228. [Google Scholar]
  33. Zheng, Z.; Yang, M.; Hong, J.; Zhao, C.; Xu, G.; Yang, L.; Shen, C.; Yu, X. DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning, 2026. arXiv.
  34. Zhang, Y.F.; Lu, X.; Yin, S.; Fu, C.; Chen, W.; Hu, X.; Wen, B.; Jiang, K.; Liu, C.; Zhang, T.; et al. Thyme: Think Beyond Images. arXiv 2025, arXiv:2508.11630. [Google Scholar]
  35. Qiao, R.; Tan, Q.; Dong, G.; Wu, M.; Sun, C.; Song, X.; Wang, J.; Gongque, Z.; Lei, S.; Zhang, Y.; et al. We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning? In Proceedings of the Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025; Vienna, Austria, Che, W., Nabende, J., Shutova, E., Pilehvar, M.T., Eds.; Association for Computational Linguistics, 27 July 2025; Volume 2025, pp. 20023–20070. [Google Scholar]
  36. Qiao, R.; Tan, Q.; Yang, P.; Wang, Y.; Wang, X.; Wan, E.; Zhou, S.; Dong, G.; Zeng, Y.; Xu, Y.; et al. We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning. CoRR 2025. abs/2508.10433, [2508.10433]. [Google Scholar] [CrossRef]
  37. Qiao, R.; Tan, Q.; Dong, G.; MinhuiWu, M.; Wang, J.; Zhang, Y.; GongQue, Z.; Sun, C.; Xu, Y.; Xue, Y.; et al. V-oracle: Making progressive reasoning in deciphering oracle bones for you and me. Proceedings of the Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics 2025, Volume 1, 20124–20150. [Google Scholar] [CrossRef]
  38. Li, Y.; Inan, H.A.; Yue, X.; Chen, W.N.; Wutschitz, L.; Kulkarni, J.; Poovendran, R.; Sim, R.; Rajmohan, S. Simulating Environments with Reasoning Models for Agent Training. arXiv 2025. [Google Scholar]
  39. Ren, J.; Zhuang, Y.; Ye, X.; Mao, L.; He, X.; Shen, J.; Dogra, M.; Liang, Y.; Zhang, R.; Yue, T.; et al. SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds. arXiv 2026. [Google Scholar]
  40. Li, Y.; Wang, H.; Qiu, J.; Yin, Z.; Zhang, D.; Qian, C.; Li, Z.; Ma, P.; Chen, G.; Ji, H.; et al. From Word to World: Can Large Language Models be Implicit Text-based World Models? arXiv 2025. [Google Scholar]
  41. Tian, X.; Wang, H.; Chen, S.; Zhou, H.; Yu, K.; Zhang, Y.; Ouyang, J.; Yin, J.; Chen, J.; Guo, B.; et al. ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas, 2026. arXiv.
  42. Wang, Z.; Xu, C.; Liu, B.; Wang, Y.; Han, S.; Yao, Z.; Yao, H.; He, Y. Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning, 2026. arXiv.
  43. Song, X.; Chang, H.; Dong, G.; Zhu, Y.; Dou, Z.; Wen, J.R. EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis. arXiv 2026, arXiv:2601.05808. [Google Scholar]
  44. Sullivan, M.; Hartmann, M.; Koller, A. Procedural Environment Generation for Tool-Use Agents. In Proceedings of the Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing; Christodoulopoulos, C., Chakraborty, T., Rose, C., Peng, V., Eds.; Suzhou, China, 2025; pp. 18544–18562. [Google Scholar] [CrossRef]
  45. Fang, R.; Cai, S.; Li, B.; Wu, J.; Li, G.; Yin, W.; Wang, X.; Wang, X.; Su, L.; Zhang, Z.; et al. Towards General Agentic Intelligence via Environment Scaling. arXiv 2025, arXiv:2509.13311. [Google Scholar]
  46. Tu, D.; Hao, H.; Yang, H.; Chen, Y.; Zhang, Y.K.; Xia, Z.; Yang, Y.; Sun, Y.; Liu, X.; Shen, F.; et al. ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training. arXiv 2026. [Google Scholar]
  47. Zhang, Z.; Wang, Z.; Zhang, X.; Guo, Z.; Li, J.; Li, B.; Lu, Y. InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training. arXiv 2026. [Google Scholar]
  48. Andrews, P.; Benhalloum, A.; Bertran, G.M.T.; Bettini, M.; Budhiraja, A.; Cabral, R.S.; Do, V.; Froger, R.; Garreau, E.; Gaya, J.B.; et al. Are: Scaling up agent environments and evaluations. arXiv 2025, arXiv:2509.17158. [Google Scholar]
  49. Doshi, R. Gemini 3 Pro: the frontier of vision AI. 2025. Available online: https://blog.google/innovation-and-ai/technology/developers-tools/gemini-3-pro-vision/.
  50. Team, K.; Bai, T.; Bai, Y.; Bao, Y.; Cai, S.H.; Cao, Y.; Charles, Y.; Che, H.S.; Chen, C.; Chen, G.; et al. Kimi K2.5: Visual Agentic Intelligence. arXiv 2026. [Google Scholar]
  51. Qwen Team. Qwen3.5: Towards Native Multimodal Agents. 2026. [Google Scholar] [CrossRef]
  52. Bai, S.; Cai, Y.; Chen, R.; Chen, K.; Chen, X.; Cheng, Z.; Deng, L.; Ding, W.; Gao, C.; Ge, C.; et al. Qwen3-VL Technical Report. arXiv 2025. [Google Scholar]
  53. Team, Q. Qwen3: Think Deeper, Act Faster. 2025. Available online: https://qwenlm.github.io/blog/qwen3/.
  54. DeepSeek-AI. DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models. 2025. [Google Scholar] [CrossRef]
  55. Bai, L.; Cai, Z.; Cao, M.; Cao, W.; Chen, C.; Chen, H.; Chen, K.; Chen, P.; Chen, Y.; Chen, Y.; et al. Intern-S1: A Scientific Multimodal Foundation Model. arXiv 2025. [Google Scholar]
  56. Sheng, G.; Zhang, C.; Ye, Z.; Wu, X.; Zhang, W.; Zhang, R.; Peng, Y.; Lin, H.; Wu, C. HybridFlow: A Flexible and Efficient RLHF Framework. arXiv 2409.19256. 2024. [Google Scholar]
Table 1. Model performance across the 10-task InternAgentHarness suite. Each task contains 100 held-out evaluation instances. We report success rate (%) for eight text-only tasks and two vision tasks, together with modality-specific averages. The final two rows show our environment-aligned SFT checkpoint and the subsequent RL checkpoint initialized from it.
Table 1. Model performance across the 10-task InternAgentHarness suite. Each task contains 100 held-out evaluation instances. We report success rate (%) for eight text-only tasks and two vision tasks, together with modality-specific averages. The final two rows show our environment-aligned SFT checkpoint and the subsequent RL checkpoint initialized from it.
Model Text-only Tasks Vision Tasks
battery fenics epidemic mole music bot finance escape Average flangeplane holdem Average
Multi-Modal LLM
Gemini-3.1-Pro-Preview-Thinking 82.0 97.0 29.0 32.0 40.3 56.0 97.3 71.0 63.1 100.0 51.0 75.5
Gemini-3.1-Pro-Preview 78.0 96.0 28.0 34.0 38.5 43.0 98.2 70.0 60.7 100.0 50.0 75.0
Kimi-K2.5 20.0 81.0 44.0 47.0 31.1 14.0 85.7 34.2 44.6 36.0 47.8 41.9
Kimi-K2.5-Thinking 29.0 75.8 37.0 38.0 36.7 6.0 88.2 28.4 42.4 45.0 51.0 48.0
Qwen3.5-397B-A17B 41.0 45.0 28.0 12.0 10.0 7.0 82.5 22.8 31.0 62.0 51.6 56.8
Qwen3.5-397B-A17B-Thinking 37.0 44.0 29.0 15.0 6.0 3.0 82.8 22.5 29.9 63.0 50.5 56.8
Intern-S1-Pro 45.0 56.0 13.0 26.0 7.5 1.0 72.3 12.0 29.1 48.0 43.6 45.8
Intern-S1 0.0 18.3 11.7 22.7 6.5 2.0 77.9 1.9 17.6 - - -
Qwen3-VL-235B-A22B-Thinking 18.0 14.3 7.0 12.0 15.5 1.0 73.8 19.2 20.1 97.0 50.8 73.9
Qwen3-VL-235B-A22B-Instruct 4.0 3.5 0.0 11.0 3.5 0.0 43.0 0.0 8.1 74.0 22.4 48.2
Qwen3-VL-30B-A3B-Thinking 16.0 0.0 0.0 0.0 0.0 0.0 12.3 0.0 3.5 77.0 0.0 38.5
Qwen3-VL-30B-A3B-Instruct 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.0 0.2 64.0 0.0 32.0
Gemini-3-Pro-Preview 24.0 89.0 27.0 51.0 40.5 65.0 95.6 78.1 58.8 80.0 35.3 57.7
Text-only LLM
DeepSeek-V3.2-Exp 16.0 45.8 22.0 41.0 24.8 20.0 78.5 19.4 33.4 - - -
Qwen3-235B-A22B-Instruct-2507 0.0 40.0 1.0 12.0 4.3 1.0 50.5 0.0 13.6 - - -
Qwen3-32B 15.0 4.5 11.0 10.0 3.5 1.0 54.3 1.8 12.6 - - -
Qwen3-8B-thinking 0.0 6.5 0.0 8.0 0.3 0.0 33.5 3.8 6.5 - - -
Qwen3-VL-30B-A3B-SFT 20.0 35.9 18.0 14.0 34.0 5.0 85.0 42.8 31.8 91.0 34.5 62.8
Qwen3-VL-30B-A3B-SFT-RL 67.0 83.3 23.0 24.0 49.0 25.0 98.4 41.1 51.4 98.0 49.8 73.9
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.