Agentic Visual Generation: A Survey

1. Introduction

From Single-Pass Rendering to Closed-Loop Visual Creation.

Visual generation has developed from a specialized media-synthesis capability into a general component of artificial intelligence infrastructure. It now supports the production and editing of images, videos, three-dimensional assets, data-grounded graphics, structured documents, interactive interfaces, and related visual artifacts across design, entertainment, education, science, industry, and other application settings. Advances in generative and multimodal models have substantially improved visual fidelity, instruction and reference adherence, text rendering, controllability, editing, and other task-specific capabilities [1,2,3,4]. A single-pass renderer can now often produce a plausible artifact from a concise instruction and a limited set of conditions.

The expansion from isolated synthesis to multi-step visual creation tasks changes the requirements placed on a visual system. A single artifact or sequence may need to preserve precise spatial relations, target identity, temporal order, editable structure, factual or data-grounded content, physical plausibility, stable interaction, and other cross-step requirements. Surveys, benchmarks, and recent system studies continue to report weaknesses along structural, temporal, physical, and other task-specific dimensions even when individual outputs achieve high perceptual quality [4,5,6,7]. Their practical consequence is cumulative: an early relation error can invalidate a later edit, identity drift can propagate through a video, an incorrect data mapping can undermine a scientific graphic, and an invalid geometric operation can make a visually plausible model unusable, among other downstream effects. Meeting these requirements calls for creation control: constraint persistence, usable intermediate state, diagnosis, repair, and other forms of trajectory management.

Model scaling, prompt refinement, additional conditioning, repeated sampling, and other output-level refinements can improve individual outputs. None of them closes the loop: a visual creation process remains open-loop when runtime observations do not change subsequent decisions. Under this condition, the system cannot use intermediate outcomes to determine whether the current plan remains appropriate, which constraints have been satisfied or violated, where a failure has occurred, or when the task should stop. Local errors and changes in task state may then persist or propagate across later edits, frames, components, and other dependent outputs.

Avoiding such accumulation requires goal-directed control over the creation trajectory. The system must maintain task-relevant state, inspect intermediate artifacts and execution outcomes, and use the resulting evidence to determine subsequent actions. Recent years show rapid progress toward this capability: since 2024, systems combining artifact observation, constraint verification, tool selection, and iterative correction have emerged across compositional image generation [8], programmatic video production [9], and multi-turn image editing [10], with early extensions toward physical generation, scientific visualization, and structured documents. These developments mark a transition from open-loop sampling toward closed-loop visual creation, in which runtime observations change later action selection.

Figure 1 quantifies this transition across the surveyed corpus; the inclusion criteria and coding protocol are described in Section 2. The number of systems expands rapidly after 2023 (panel a), and the growth spans visual domains: closed-loop designs now appear across images, video, three-dimensional assets, data-grounded graphics and documents, and interactive interfaces (panel c), so the same control questions recur under different representations, constraints, and sources of verification evidence. The composition by autonomy level shifts upward in parallel, with a growing share of systems reporting behaviors beyond fixed pipelines (panel d), as graded by the L1–L5 levels introduced in Section 2; panel b further shows how these evidence-supported levels vary across visual domains. Yet the aggregate trend also exposes the gap behind the growth: the designs remain scattered across application papers, autonomy claims rest on heterogeneous evidence, and no unified, control-centered account exists of what makes such systems agentic, how they should be organized, or how they should be evaluated.

In this survey, we develop that account. We name this class of systems agentic visual generation (AVG) and present its working definition below.

Figure 2 situates this behavioral boundary within a broader evolution of visual-generation control. The progression moves from fixed condition-to-artifact mappings toward increasingly broad forms of state-dependent control: runtime tool selection, feedback-driven adaptation, long-horizon replanning and recovery, and persistent cross-task improvement. The levels are cumulative behavioral categories rather than strictly chronological stages; the timeline locates selected representative systems historically.

Review Motivation and Questions.

Existing related surveys examine adjacent questions from different starting points. Visual-generation surveys organize the literature around model families, conditioning and editing methods, or particular visual tasks and modalities [2,3,4]. The survey of He et al. studies how language models participate in multimodal generation and editing, including through tool-mediated interaction [1]. Agent surveys instead develop concepts, capabilities, and system designs for language agents or multimodal agents across broad task settings [11,12,13,14]. At a different level of abstraction, the recent visual-generation roadmap situates agentic generation within a longer progression toward visual intelligence and world modeling [5].

These reviews provide essential foundations, but their primary questions differ from the one considered here. To the best of our knowledge, no existing survey is devoted specifically to agentic visual generation as a class of systems in which runtime observations demonstrably change subsequent visual-creation decisions. This survey uses that behavioral property to organize the literature, then compares how the resulting control mechanisms and supporting evidence vary across visual-creation domains.

The survey addresses six research questions:

1.

Conceptual boundary: What constitutes AVG, and how does it relate to one-shot generators, fixed workflows, and agent-augmented workflows? (Section 2)

2.

Analytical components: Which components organize the connection between runtime observations and subsequent visual-creation decisions? (Section 3)

3.

Cross-domain variation: How are these mechanisms realized across visual-creation domains? (Section 4, Section 5, Section 6, Section 7, Section 8, Section 9 and Section 10)

4.

Training and improvement: How are AVG systems trained and improved through trainable components, visual trajectories, feedback, experience reuse, and cross-task transfer? (Section 11)

5.

Evaluation: How should the outcomes and process control of AVG systems be evaluated? (Section 12)

6.

Research frontier: Which open problems and evidence requirements define future AVG research? (Section 13)

Contributions and Organization.

This survey makes six contributions:

1.

It defines and formalizes agentic visual generation as closed-loop visual creation and specifies a behavioral boundary based on observation-dependent creation decisions.

2.

It synthesizes a six-component analytical framework comprising goal understanding, specification, and planning; memory; tool; perception; action; and cross-task self-improvement.

3.

It compares how these mechanisms operate across visual-creation domains with different representations, constraints, action spaces, sources of verification evidence, and other domain-specific properties.

4.

It analyzes training and improvement in AVG systems, from trainable targets and visual trajectory supervision to policy initialization, reinforcement learning from multimodal and executable feedback, and experience reuse with skill abstraction, joint optimization, and cross-task transfer.

5.

It organizes evaluation evidence into four complementary levels (artifact quality, goal and constraint satisfaction, trajectory and decision quality, and system and human-centered evaluation) and identifies the remaining gaps in current evaluation practice.

6.

It synthesizes research frontiers for agentic visual generation around expanding the scope of visual agency, increasing the intelligence of visual agents, and establishing reliable evaluation.

Figure 3 provides a cross-domain overview of the representative systems and benchmarks examined in Section 4, Section 5, Section 6, Section 7, Section 8, Section 9 and Section 10.

The remainder of the survey follows this control-centered organization. Section 2 introduces visual-generation foundations, defines and formalizes AVG, establishes its boundary, and presents the L1–L5 autonomy levels. Section 3 examines the six analytical components that organize visual-agent systems. Section 4, Section 5, Section 6, Section 7, Section 8, Section 9 and Section 10 compare their realization across visual-creation domains and synthesize the domain evidence. Section 11 examines trajectory supervision, policy initialization, multimodal reinforcement learning, experience reuse, and transfer. Section 12 organizes evaluation evidence into four complementary levels and identifies remaining evaluation gaps, Section 13 examines research frontiers, and Section 14 concludes the survey.

2. Foundations of Agentic Visual Generation

2.1. Visual Generation Foundations

Visual generation in this survey covers the construction and editing of images, videos, three-dimensional scenes, parametric CAD models, scientific graphics, structured documents, interactive interfaces, and related visual products. Four foundations connect these operations to agentic control: the generator, the artifact representation, the conditioning inputs, and the execution interfaces. Together, they determine how an artifact is produced, what task-relevant information persists, what information an operation can use, and what actions the system can execute.

2.1.1. Visual Generators

In this survey, a visual generator is a model that maps a condition to a visual artifact or an editable visual representation. For a stochastic generator, the output x can be written as a sample from a conditional distribution $p_{\varphi }(x\mid c)$, where c may contain text, images, spatial references, structured requirements, or information retained from an earlier operation. The main research question is therefore how a model represents x, how it parameterizes the conditional distribution, and which parts of c can be used to control the result. The history of visual generators is best understood as a sequence of changes in representation and conditioning, with image and video models providing the main large-scale foundations and specialized systems adapting these foundations to structured outputs [1,2,3,4,5]. Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8 organize this development by output type, and each is discussed in the corresponding paragraph below.

Image foundation models.

The first generation of neural image models focused on learning a data distribution from which new samples could be drawn. Variational autoencoders introduced an explicit latent-variable model with approximate inference, while generative adversarial networks learned a generator through a competition between a generator and a discriminator [137,138]. Normalizing flows added invertible transformations and tractable likelihood evaluation [139,140]. In each case, training estimates a distribution over visual observations and inference samples from the learned distribution. The choice of latent variables, discriminator features, or invertible coordinates affects diversity, sharpness, likelihood estimation, and the ease with which a condition can be injected. These models established the separation between a representation space and a sampling procedure, and they made learned visual synthesis possible on data distributions that are difficult to specify with hand-written rules. Their interfaces were still mainly designed for unconditional sampling or a small set of manually designed conditions, so precise spatial control and reliable local editing were not yet central properties of the generator.

Autoregressive models then made the order of visual prediction explicit. PixelRNN predicted pixels sequentially, turning image synthesis into a conditional sequence model whose next prediction depends on the preceding context. VQ-VAE and VQGAN learned discrete visual representations that allowed an image to be treated as a sequence of tokens, and DALL-E used such a representation for large-scale text-conditioned generation [141,142,143,144]. The tokenizer became an important part of the generator: its codebook determines which visual details are retained, while the Transformer models dependencies among the resulting tokens. MaskGIT changed the decoding schedule by masking groups of tokens and filling them in parallel over several refinement steps, trading a strict left-to-right order for iterative global refinement [145]. Visual Autoregressive Modeling (VAR) organized prediction from coarse to fine scales, reducing the effective sequence length through next-scale prediction [146]. LlamaGen and MAR revisit the same family at larger scale, with MAR modeling continuous token distributions together with a diffusion objective [147,148]. This line of work therefore changes both the object being generated and the way generation proceeds: pixels become compact visual units, and a single pass becomes an ordered or iterative prediction process. The representation can be inspected, masked, reordered, or extended, which later supports token-level conditioning and editing.

Diffusion models formed a second major route for image foundation models. DDPM defined generation as the reversal of a gradual noising process: the model learns to remove noise at a sequence of noise levels, and sampling follows the learned reverse transitions from a noisy state to an image. Score-based formulations described the same family through the score of perturbed data distributions and connected discrete denoising to continuous stochastic differential equations [149,150]. Latent diffusion moved denoising into a compressed representation, separating perceptual compression from the generative process and making high-resolution synthesis more practical. Imagen demonstrated the effect of stronger language conditioning, while DiT replaced the convolutional denoiser with a Transformer operating on latent patches [151,152,153]. Flow matching and rectified flow learn continuous transport paths between source and target distributions, and consistency models target one-step or few-step sampling by enforcing agreement among denoising trajectories [154,155,156]. These formulations expose several controllable interfaces: the condition can enter at every denoising step, the latent representation can be edited before decoding, and the sampler can be selected according to a quality or latency requirement. They consequently provide the backends used by many later editing and agentic systems, while the number of sampling steps and the quality of the learned condition remain important practical variables. On the open-weights side, FLUX.1 scales rectified flow to a twelve-billion-parameter transformer and stands as the leading publicly available image generator of its generation [157].

The most recent generation of image foundation models combines the two preceding routes within a single system. An autoregressive Transformer first predicts discrete image tokens conditioned on text and reference images, so the request is interpreted with the reasoning and instruction-following habits of language models; the hidden states of this backbone then serve as the condition for a diffusion model that renders the final image in continuous latent space, inheriting the rendering quality of denoising. BLIP3o-NEXT identifies this composition as the current default for open-weight native image generators [158]. HunyuanImage 3.0 scales the recipe to an 80-billion-parameter mixture-of-experts language model with chain-of-thought supervision [159], Qwen-Image pairs a vision-language condition encoder with a multimodal diffusion transformer for paragraph-level text rendering [160], and FLUX.2 combines a 32-billion-parameter rectified-flow transformer with a 24-billion-parameter vision-language encoder [161]. Reinforcement-learning post-training is applied on top of this composition, transferring the policy-optimization infrastructure of large language models to image generation and improving text rendering and instruction following beyond supervised finetuning [162]. In this setting, text, reference images, and spatial signals such as masks, edges, and depth enter through a single multimodal conversation, and operations that previously required separate editing pipelines reduce to a prompt. Figure 4 contrasts the five paradigms by latent representation, transport trajectory, and degree of iterative refinement.

Video foundation models.

Video generators extend image generation with a temporal representation: an output is a sequence whose frames must remain individually plausible while preserving motion, identity, and scene continuity, so the generator models correlations across space and time under conditions that may specify an initial frame, a motion pattern, a camera change, or a textual event. Early attempts used video GANs, recurrent predictors, and 3D convolutions, but they produced only seconds-long, low-resolution clips because adversarial objectives remained difficult to stabilize over high-dimensional spatiotemporal data [163,164].

The field shifted decisively in 2022 along two parallel routes. The autoregressive route modeled video as ordered discrete tokens: CogVideo scaled this idea to a nine-billion-parameter transformer and became the first large-scale open-source pretrained text-to-video system, while Phenaki coupled a causal video tokenizer with a transformer to generate minute-level sequences from chained prompts [165,166]. The diffusion route transplanted image denoisers to spatiotemporal data: Video Diffusion Models established the core formulation with factorized noise trained jointly on images and videos [167], Make-A-Video showed that a text-to-image prior can supply appearance while unlabeled video supplies motion, avoiding paired text-video supervision altogether [168], and Imagen Video composed seven cascaded diffusion stages for progressive refinement [169].

Work from 2023 onward consolidated around latent diffusion with explicit temporal modules. ModelScopeT2V combined a text encoder, VQGAN, and a denoising UNet with spatiotemporal blocks into a widely replicated design [170]. Instead of learning appearance and motion jointly, systems reused frozen text-to-image backbones and inserted lightweight temporal layers: AnimateDiff added a plug-and-play motion module, Stable Video Diffusion specialized a latent backbone for image-to-video synthesis, and Emu Video factored text-to-video into text-to-image followed by image-to-video [171,172,173]. This strategy reduced training cost substantially while inheriting the prompt-following behavior and visual diversity of strong image priors.

A second inflection arrived with OpenAI’s Sora, which replaced convolutional denoisers with a diffusion transformer operating on spacetime patches at native resolution and duration, making a single scalable architecture handle still images, short clips, and extended sequences interchangeably [174]. Foundation-scale systems followed in quick succession: Movie Gen paired a thirty-billion-parameter video model with a dedicated thirteen-billion-parameter audio model for joint generation, CogVideoX unified spatial and temporal modeling inside a single 3D full-attention block as the open-source successor to CogVideo, and HunyuanVideo and Wan advanced open-source alternatives with larger corpora, temporal compression, and efficient inference [175,176,177,178]. Their interfaces now span text-to-video, image-to-video, video extension, camera trajectories, and reference-conditioned editing, shifting the focus from rendering isolated frames toward modeling a temporally persistent visual state in which objects survive, cameras move, and actions unfold across many seconds. Joint audio-visual generation marks the current frontier: Veo 3 produces synchronized dialogue, effects, and ambient sound alongside frames by processing visual spacetime patches and temporal audio channels within one diffusion pass [179], and recent releases across proprietary and open-source families continue to push duration, resolution, and prompt adherence upward. Figure 5 traces this progression from early spatiotemporal extensions to foundation-scale audio-visual systems.

3D and world foundation models.

Three-dimensional generators use representations such as voxels, point clouds, meshes, neural fields, or Gaussian primitives, which differ in how they encode geometry, appearance, topology, and view-dependent effects, and determine whether a generated object can be rendered, measured, or edited after synthesis. NeRF first showed that a continuous neural field maps spatial locations and viewing directions to density and color, making a scene renderable from new viewpoints [180]. DreamFusion then used a pretrained two-dimensional diffusion model as a prior and optimized a three-dimensional representation through repeated rendering, establishing the influential route to text-to-3D when large paired 3D datasets were scarce, at the cost of slow per-instance optimization and sensitivity to multi-view consistency [181].

The following stage emphasizes faster reconstruction and more explicit scene representations. 3D Gaussian Splatting represents a scene with oriented, colored Gaussian primitives for real-time radiance-field rendering [182], while LRM, TripoSR, VFusion3D, and Hunyuan3D replace per-instance optimization with learned feed-forward mappings from image or multi-view-diffusion evidence to 3D assets [183,184,185,186]. TRELLIS extends this line toward editable structure: its structured latents decode a single generated object into radiance fields, Gaussians, or meshes, support tuning-free local editing, and scale to rectified-flow transformers with up to two billion parameters trained on 500K objects [187]. Together, these systems make 3D generation suitable for repeated calls inside larger workflows, although geometric completeness and cross-view agreement remain central quality dimensions.

The current frontier extends from individual assets to scenes, dynamic environments, and interactive worlds along three lines of work. Video-trained world models generate action-controllable environments: Genie 2 turned a prompt image into keyboard-playable scenes persisting for tens of seconds, and Genie 3 sustains real-time navigation at 720p and 24 fps over several minutes [188,189]. Physics-aware world foundation models target embodied use at scale: Cosmos couples large-scale pretraining with future-state video prediction for robots and autonomous vehicles and releases early checkpoints under an open license [190]. Diffusion-based neural game engines learn interactivity end to end: GameNGen simulates DOOM interactively above 20 frames per second on a single TPU by conditioning next-frame generation on recorded actions, and the open-source Oasis demonstrates real-time Minecraft-style play [191,192].

A world-oriented generator must preserve spatial relations across objects, support changes over time, and define how an observation, an action, and a subsequent visual state are related, properties that distinguish a world model from a generator that produces a single plausible object. Figure 6 contrasts these routes by representation explicitness, reconstruction cost, and whether the output couples observation, action, and subsequent visual state.

Structured and executable visual generators.

Some visual tasks require a representation that remains editable or executable after rendering. In these tasks, the generator outputs a layout tree, a visualization specification, a construction sequence, or source code alongside the rendered view, so that the produced artifact can be inspected, modified, and re-rendered rather than delivered only as a final appearance. This distinguishes the generators in this paragraph from those in the preceding ones: an image or video model terminates at rendered pixels, whereas a structured generator preserves the internal construction that produced those pixels, which is what allows a later operation to target a specific component, parameter, or code block. The development of this family (covering layout, chart, document, CAD, and UI generation) progressed through four stages, drawing on research communities that worked largely in parallel before converging in the LLM era.

Early systems came from the human–computer interaction community and relied on templates, visual grammars, and component libraries, with hand-designed rules guaranteeing structural validity at the cost of coverage bounded by the available templates and production rules [193]. The neural stage arrived with sequence-to-sequence translation from the NLP community: Data2Vis mapped data specifications to Vega-Lite visualization programs with an attention-based LSTM [194], and pix2code generated platform-specific interface code directly from a GUI screenshot with over 77% accuracy across iOS, Android, and web targets [195], establishing that structured output can be learned end to end rather than authored.

Work from 2021 onward turned to construction sequences, replacing a final artifact with a program whose rendering is repeatable and editable. DeepCAD trained a Transformer-based autoencoder on 178,238 Onshape models to generate sketch-and-extrude sequences that preserve construction history rather than only a final mesh [196]; SkexGen disentangled topology, geometry, and extrusion into separate codebooks for finer control [197]; and LayoutDM handled controllable layout generation across multiple tasks with modality-wise discrete diffusion [198]. Because the generated object is a structured description rather than a rendered view, a later operation can inspect the construction history, modify a step, and re-render, associating a visible defect with a source-level element.

The current stage connects general-purpose language and multimodal models to these structured representations, and in doing so merges the previously separate communities. A model can produce HTML/CSS/JavaScript, a chart specification, or a CAD program and hand the result to a renderer, parser, or execution environment: LIDA generates grammar-agnostic visualizations from natural-language summaries of data [199]. Benchmarking efforts such as Design2Code evaluate multimodal models on converting webpage screenshots into code, and find that models mostly lag in recalling visual elements and generating correct layouts [200]; Sketch2Code extends this line to low-fidelity hand-drawn sketches and further shows that even the strongest models struggle to interpret sketches and to formulate clarifying questions in multi-turn interaction [201]. Across structured domains, the long-term trend is from rendering a finished appearance to generating an editable or executable object whose consequences can be observed, with each community contributing a distinct structural prior that LLM-era systems now inherit together. Figure 7 depicts the four stages and the structural priors each contributed.

Unified multimodal visual generation models.

The development of visual generators also includes a cross-modal route that unifies visual understanding and generation. Unlike the preceding paragraphs, which are organized by the type of visual output, this line is organized by how understanding and generation are coupled within a single model. Its first stage keeps modality-specific encoders and decoders around a shared language or Transformer representation: text is encoded into a condition, the visual generator produces an artifact, and a separate vision encoder interprets the result. The interfaces are clear, but information must be translated whenever the system moves between understanding and generation. Mixed-token models reduce this separation by placing text and visual content in one sequence: Chameleon and Emu3 model mixed visual–textual sequences, and Show-o combines autoregressive prediction with discrete diffusion so that the same backbone can interpret context and produce visual content [202,203,204]. A new visual prediction can now be conditioned on an earlier image, a textual explanation, and preceding visual tokens without constructing a task-specific interface for each transition, which broadens the information a generator can consume and lets generation draw on visual context, intermediate representations, and language-level reasoning.

The next stage introduces specialized tokenizers or encoding paths within a unified system, because understanding favors features that preserve semantic distinctions while generation requires representations that retain appearance and permit high-quality decoding. Janus separates visual encoding paths for understanding and generation while retaining a unified multimodal interface [205]. This organization supports image-conditioned generation, visual question answering followed by editing, and interleaved multimodal output, with trade-offs among understanding accuracy, generation quality, tokenization choices, and inference cost. The pathway choice also determines which intermediate representation can be passed to a later operation: a shared sequence can retain multimodal context, while a specialized generative latent can support efficient rendering.

The current frontier carries this route into production-scale systems. GPT-4o generates images natively inside the language model: an autoregressive decoder interleaves text tokens with continuous image latents that a diffusion head decodes, so the same conversational context drives both dialogue and synthesis, yielding accurate in-image text rendering, detailed multi-object instruction following, and edits that build on previous turns [206]. Gemini 2.5 Flash Image extends the same interface toward editing: it pairs multi-image fusion with character and style consistency across generated sequences and led the LMArena image-editing leaderboard at release, turning reference-preserving, conversation-driven editing into a first-class capability rather than a mask-driven pipeline [207]. Across these systems the condition space converges on multimodal conversation itself, where reference images, instructions, and dialogue history jointly determine what is generated. Unified multimodal modeling is a foundation-model direction: it determines what a single model can consume and produce. Figure 8 contrasts the four coupling stages.

2.1.2. Visual Artifacts and Task Evidence

A visual artifact is the object that a system creates, edits, or delivers. Common forms include rendered images and videos, structured scene or geometric representations, data-grounded graphics, documents, interactive interfaces, and related artifacts. Depending on its representation, an artifact may contain rendered appearance together with temporal relations, geometry, parameters, source data, hierarchy, or interaction state. These representations can preserve task-relevant properties across operations [4,180,194,195,196].

The same visual result can be carried by different representations. A poster, for example, may exist as a raster image, as a layered graphics file, or as a parameterized template: the rendered appearance is identical, yet a raster image supports only appearance-level edits, the layered file allows object-level changes, and the template turns a single parameter change into a global update. An artifact is therefore characterized by the representation in which it persists between operations, not only by its rendered appearance.

The artifact representation, together with the task requirements, helps determine what constitutes a valid result. Video tasks may require temporal preservation of identities, events, motion, camera changes, and other sequence-level properties. Three-dimensional and CAD representations can expose geometry, topology, parameters, and construction constraints. Data-driven graphics may retain links to source data, structured documents may retain layout or hierarchy, and interfaces may retain executable behavior. Other visual products impose their own representation-specific requirements and sources of evidence.

Across these forms, validity requirements group into a few recurring kinds. One kind asks whether the artifact faithfully realizes the condition: whether an edited image follows the instruction while preserving the rest, or whether a generated page matches the request. A second kind asks whether the artifact is internally consistent: whether video frames cohere into a continuous sequence, whether a mesh closes into a manifold, or whether cross-references inside a document resolve. A third kind checks the artifact against external references: whether a chart’s plotted values match the underlying table, whether a generated model respects the specified dimensions, or whether an interactive page behaves correctly when exercised. Each kind calls on different evidence.

Correctness therefore depends on evidence matched to the representation and the task, such as temporal checks, geometric or constraint checks, data consistency, layout inspection, or functional execution. A representation also sets the upper bound on what can be verified: pixel-only artifacts admit appearance comparison, while representations that retain construction, structure, or code admit structural and functional checks as well. The representation and the available operations together determine which properties can be inspected, preserved, or modified later.

2.1.3. Conditioning Inputs

A conditioning input is information supplied before a particular generation or editing operation. It helps specify what that operation should create, preserve, or change. The main forms are textual instructions, visual or spatial references, and structured task information. Text provides semantic intent and requested content; visual or spatial references provide appearance, identity, location, or layout cues; structured information provides explicit data values, geometric constraints, programmatic specifications, or other task requirements [2,3,4].

The three forms differ in what they fix and what they leave open. Text may state a complete request or a single attribute to change, and the same words can carry an intent to realize, a style to imitate, or a constraint to respect. A visual reference likewise plays different roles: the same image may serve as content to preserve, an identity to transfer, a composition to follow, or the canvas of an edit. Structured information is the most explicit form: a data table, a schema, a layout tree, or a program pins down values and relations that text describes only loosely. A condition is thus a partial specification of the task.

The role of a condition depends on the operation that receives it. An initial operation may use a user request and reference material, while a later operation may use the current artifact, a selected region, an intermediate specification, or a verified state. These inputs make some task variables explicit while leaving other requirements to be inferred, clarified, or checked after execution. They also affect the granularity of control, which follows from both the form of the condition and the representation of the artifact: a text prompt acts on the whole output, a mask or a box restricts the change to a region, a reference to an object addresses one element of a structured scene, and a single parameter touches one construction step; the latter granularities exist only when the artifact representation retains the corresponding structure.

Over a sequence of operations, the composition of conditions shifts. Early operations are driven mainly by user-supplied conditions; as the artifact takes shape, later operations increasingly consume what the system itself has produced, and the original request turns from a primary driver into a background constraint on everything that follows. Several conditions also routinely enter one operation together, each constraining a different dimension: an instruction states what to change, a region states where, a reference image states how the result should look, and structured values state what must remain exact.

2.1.4. Execution Interfaces

An execution interface is the callable boundary through which a controller invokes an operation on an artifact or task environment, or obtains task-relevant execution evidence. It defines the callable operations, accepted inputs, returned outputs, and execution status or error information made available to the controller. For a structured artifact, an operation may alter a representation or render it; for an interactive interface or other executable artifact, it may run code, tests, or interaction steps. Interfaces differ in operation granularity and feedback channels. They connect a controller decision to an artifact or environment update, or to task-relevant execution evidence [1,4]. A controller never touches the artifact directly: every change it makes passes through such a boundary, and every observation it receives is whatever the boundary returns.

The operations an interface can offer are bounded by the representation beneath it. Pixel-level artifacts admit regeneration and appearance edits; layered or structured artifacts admit object- and parameter-level selection and modification; executable artifacts admit running, testing, and debugging. What the interface returns is subject to the same bound. Beyond the updated artifact, an interface can report status and diagnostics (which step failed, or which constraint was violated), and this localization is what allows a later operation to target the cause rather than the symptom.

Conditions specify the information and requirements for an operation, whereas execution interfaces specify how that operation can be carried out. The available operations and their granularity determine whether a controller can make a global change, edit a local region or object, modify a parameter, call a verifier, or invoke a recovery operation. Their outputs determine what the controller can observe after execution, including a new artifact, a rendering, a structured result, an error, or a test outcome. Together, these properties define the action space and the execution evidence available at each point in a visual creation process.

2.2. Definition of Agentic Visual Generation

2.2.1. Behavioral Definition

Agentic visual generation is identified by how a system controls a visual creation trajectory. The controller faces recurring decision points: which operation or tool to invoke, which repair target to address, whether to continue or stop, and related decisions that range from clarification and planning to verification, revision, recovery, and termination [11,12,13,14]. The creation process operates through a closed loop when runtime observations of the evolving artifact, its task environment, and the interaction history change what the system selects at these points.

The formal definition is:

Agentic visual generation (AVG) is goal-driven visual creation organized as a closed loop over a multi-step trajectory. At the decision points exposed to the controller (which operation or tool to invoke, which repair target to address, whether to continue or stop, and related decisions), the system selects among available alternatives based on runtime observations drawn from accessible, task-relevant state spanning the evolving visual artifact, its task environment, and the interaction history, which is in turn updated by intermediate artifacts, execution outcomes, external information, user feedback, and other task-relevant sources.

The defining behavioral criterion is therefore a demonstrable observation–action dependency: different runtime observations lead to different selected actions. The next subsubsection formalizes this criterion as a condition on the action distribution and states the operational test used throughout the survey.

The definition is applied by examining a system’s creation trajectory. The task concerns the creation, editing, or construction of a visual artifact or an editable visual environment. An identifiable decision process faces the decision points above and retains task-relevant state in a form accessible at later decision points. This state is updated by intermediate artifacts, execution outcomes, external information, user feedback, and other task-relevant sources, and observations drawn from it are shown to change a later action selection or parameterization. The decision process may be implemented by a single model, a learned policy, a search procedure, or a hierarchy of specialized agents; the same trajectory-level criterion applies across these implementations.

2.2.2. Formalization of Closed-Loop Visual Creation

The four foundations above specify what can be produced, what representation is retained, what information is supplied to an operation, and how that operation is invoked. We abstract a visual-creation trajectory using these same elements. Let g denote the task goal and its requirements, $x_t$ the current artifact representation, $s_t$ the relevant state of the task environment, $h_t$ the accessible interaction history up to step t, and ${\mathcal{I}}_t$ the execution interfaces available at step t. The triple $(x_t,s_t,h_t)$ instantiates the accessible, task-relevant state of the definition: the evolving visual artifact, its task environment, and the interaction history. We write $z_t=(x_t,s_t,h_t)$ for this joint accessible state. The artifact representation $x_t$ may be a rendered image or video, a structured scene or geometric model, a document or layout, executable interface code, or another representation appropriate to the task. The environment state $s_t$ includes information needed to continue execution, such as application state, source data, files, or execution results. The history $h_t$ records prior actions, observations, outcomes, and user interactions. The formulation instantiates a POMDP whose information state is $z_t$: because $h_t$ carries the full interaction history, the state needed for the next decision is fully contained in $z_t$, and the policy conditions only on g, on quantities derived from $z_t$, and on ${\mathcal{I}}_t$.

Each step consists of a read, a decision, and a write. The read produces the runtime observation available to the controller:

$$o_t=\mathsf{O}(g,x_t,s_t,h_t).$$

(1)

The dependence on g expresses that reads from the state are task-relevant. The observation $o_t$ may contain a visual inspection, a structured property, an execution result, a constraint check, external information, or user feedback, depending on the artifact and task. The decision, conditioned on the observation, selects among the available alternatives and parameterizes the selected action. When the action invokes an execution interface, it can be written as $a_t=(i_t,c_t)$, where $i_t\in {\mathcal{I}}_t$ is the selected interface and $c_t$ is the condition and operation input supplied to it. The action can also be a control decision such as revising the plan, recovering from an error, or stopping:

$$a_t\sim {\pi }_{\theta }\left(·\mid g,o_t,h_t,{\mathcal{I}}_t\right).$$

(2)

The support of ${\pi }_{\theta }$ corresponds to the available alternatives: the callable interfaces in ${\mathcal{I}}_t$, their parameterizations, and the control decisions. Because $a_t$ includes both the interface and its condition, the observation–action dependency covers action selection and parameterization alike.

The write applies the selected action to the state through the execution interfaces. The interface set ${\mathcal{I}}_t$ is the callable boundary described above: it determines which operations can be invoked, which conditions they accept, and which artifact updates, execution outcomes, or status information they return. When $a_t=(i_t,c_t)$ invokes a stochastic generator, its output can be written as $x_{t+1}\sim p_{\varphi }(·\mid c_t)$ for a new artifact. For editing and structured construction, the same interface updates an existing representation. Both cases are represented by a Markov-kernel transition

$$(x_{t+1},s_{t+1},h_{t+1},{\mathcal{I}}_{t+1})\sim \mathsf{U}\left(·|g,x_t,s_t,h_t,a_t;{\mathcal{I}}_t\right),$$

(3)

in which the artifact channel $p_{\varphi }(·\mid c_t)$ supplies the stochastic component, and the remaining bookkeeping, including the update of $s_t$ and $h_t$ and the recording of any evidence returned by the interface, is deterministic given the drawn artifact. The kernel additionally returns the next interface set ${\mathcal{I}}_{t+1}$, allowing interfaces to become available, fail, or be retired as the artifact evolves; when the interface set is static this component is the identity. The process terminates when the controller selects a stopping decision; stopping acts as an absorbing transition after which no further observations are drawn, and the criterion below is evaluated over the steps before termination.

The write feeds the next read: the updated artifact and environment produce new runtime observations, which are available to the next decision. This is the write side of the definition: sources update the state, and the updated state enters the next observation through $\mathsf{O}$. When an evidence-producing check is invoked through ${\mathcal{I}}_t$, its result is recorded in $s_{t+1}$ or $h_{t+1}$ and becomes part of $o_{t+1}$. Its output can be visual, symbolic, geometric, data-based, executable, or interaction-based. Verification is thus tied to the representation and task requirements established in the foundation section.

The trajectory is closed-loop when runtime observations change later action selection. Because $o_t=\mathsf{O}(g,x_t,s_t,h_t)$ is determined by the accessible state, two distinct observations at the same step arise only when two accessible states $z_t=(x_t,s_t,h_t)$ and $z_t^{\prime }=(x_t^{\prime },s_t^{\prime },h_t)$ share the history component but project to different observations. Holding the task, the available interfaces, and the prior trajectory fixed, the defining condition is

$$\begin{array}{cc}\hfill \exists t,h_t,z_t\ne z_t^{\prime }\mathrm{with}& \mathsf{O}(g,z_t)\ne \mathsf{O}(g,z_t^{\prime })\hfill \\ \hfill \mathrm{and}& \mathrm{TV}\left({\pi }_{\theta }\left(·|g,\mathsf{O}(g,z_t),h_t,{\mathcal{I}}_t\right),{\pi }_{\theta }\left(·|g,\mathsf{O}(g,z_t^{\prime }),h_t,{\mathcal{I}}_t\right)\right)>0,\hfill \end{array}$$

(4)

where $z_t$ and $z_t^{\prime }$ range over accessible states reachable under the fixed task, and $\mathrm{TV}$ denotes total-variation distance; equivalently, the two action distributions differ on some measurable set of actions.

The criterion quantifies observation dependence: a policy satisfies it whenever its action distribution varies with the observation, and the condition therefore holds for a lookup table over observations and for a task-serving controller alike. Task relevance is carried by the remaining components of the framework: the goal-driven requirement of the behavioral definition rules out observation-dependent but task-irrelevant policies at the semantic level, and the evaluation section measures task relevance through constraint satisfaction and outcome quality. A utility-based strengthening brings task value into the criterion itself: it compares ${\pi }_{\theta }$ with its observation-blind marginal ${\overline{\pi }}_{\theta }(·\mid g,h_t,{\mathcal{I}}_t)$, obtained by marginalizing the observation out of ${\pi }_{\theta }$, and requires strictly higher expected task utility under the observation-conditioned policy. The strengthened form presupposes a task utility shared across the survey’s visual-creation tasks.

Three procedures verify the condition: white-box inspection of the policy’s dependence on the observation; controlled re-execution, which varies the observation while holding the goal, the history, and the interface set fixed; and a statistical test over logged trajectories whose null hypothesis is the observation-blind policy. The statistical test evaluates the action distribution: under its null hypothesis, variation across runs arises from sampling alone, and a single run shows only this sampling variation.

2.3. Boundary of Agentic Visual Generation

The boundary separates related visual-creation systems by their control paths. The formalization in Section 2.2 states the criterion: a control path is agentic when two accessible states with the same history but different observations induce different action distributions. The criterion locates a system on a spectrum, and three dimensions organize that spectrum: who fixes the sequence of operations, what evidence enters each decision, and what each decision updates. Table 1 lists representative system types along this spectrum.

Control-path authorship.

The first dimension concerns who fixes the sequence of operations. A developer may prescribe the sequence in code, in which case the controller executes a schedule determined before execution. Alternatively, the controller may select the next operation at runtime from the available alternatives, in which case the sequence emerges from the decisions themselves. The distinction parallels the workflow–agent separation in the agent literature [11,12]: workflows orchestrate calls through predefined code, while agents choose the next call from the evolving task state.

Source of runtime evidence.

The second dimension concerns what enters a decision. Static conditions include the initial request, preset parameters, and configuration chosen before execution. Runtime observations are drawn from the accessible state $(x_t,s_t,h_t)$ through the observation map $\mathsf{O}$ and may contain a visual inspection, a structured property, an execution result, a constraint check, or user feedback. A system that conditions every decision on static conditions alone carries no runtime evidence channel; a system whose decisions read from the accessible state possesses one, and the closed-loop criterion further requires that the channel influences action selection.

Reach of state update.

The third dimension concerns what each decision changes. Updates may remain within the current trajectory, affecting the artifact and its environment state; or they may persist across trajectories, changing reusable memory, skills, verifiers, policies, or models. Within-trajectory updates support recovery and replanning inside a single task; cross-trajectory updates carry experience from one task to the next.

Boundary rules.

The dimensions generate four rules for classifying a system.

First, the criterion applies at the decision points exposed to the controller: the choice of interface and condition, and the control decisions between operations. Mechanisms inside a single invoked operation, such as guidance signals or sampler adaptation within one generation call, belong to that operation. A system whose observation-driven mechanisms lie only inside individual operations falls under the one-shot generator type.

Second, agentic status is a property of the deployed control path. Training-time feedback shapes the policy through gradient updates, preference data, or trajectory supervision; the resulting generator, deployed as a single conditional sampler, carries no runtime observation channel. The loop that produced the policy belongs to the training procedure.

Third, scheduled verification and reactive verification occupy different positions on the boundary. A pipeline that runs a verifier at a fixed stage and forwards its output downstream regardless of outcome applies verification on a schedule; the verification result travels as a fixed input to the next stage. A system whose verifier outcome determines the next action, such as whether to retry, which candidate to select, or when to stop, uses verification as runtime evidence and satisfies the closed-loop criterion.

Fourth, the criterion has a minimal form. Selection among sampled candidates based on runtime evidence satisfies the criterion when the selection depends on the drawn artifacts; it constitutes the shortest form of closed-loop control. Iterative self-refinement, in which the system critiques its own output and conditions the next revision on the critique, occupies the same position when the critique enters the revision decision [208,209].

The five types mark characteristic combinations of the three dimensions. One-shot generators prescribe the sequence in advance, condition every decision on the initial request, and take a single step from the initial conditions to the artifact. Fixed workflows prescribe the sequence and pass intermediate outputs between stages; the outputs travel along a fixed route, and no decision reads from what they contain. Agent-augmented workflows prescribe the sequence and expose limited choice points; the alternatives at each point are fixed by design, and the choice may or may not depend on runtime observations. Closed-loop visual agents read from the accessible state at their decision points, and those decisions change later creation actions; the operating sequence typically emerges from the decisions, and the minimal form above retains a prescribed schedule. Continually improving agents couple closed-loop control inside a task with cross-trajectory updates to reusable components.

2.4. Autonomy Levels

AVG systems can support different spans of state-dependent control. We use five cumulative levels to record the strongest behavior supported by the available evidence, from a fixed baseline to cross-task improvement. Table 2 records the five levels and their operational criteria.

L1 is the fixed, non-adaptive baseline. L2 exposes an action space through tool, model, or role selection. L3 requires an observation–action dependency (Section 2.2): intermediate observations change one or more subsequent decisions. L4 extends this dependency across multiple operations through persistent state, replanning, recovery, and stopping. L5 additionally requires transfer: experience from earlier tasks changes later behavior under held-out evaluation. The levels and the boundary types in Table 1 record the same progression at different granularity: one-shot generators and fixed workflows sit at L1, agent-augmented workflows at L2, closed-loop visual agents at L3 or L4, and continually improving agents at L5; a system qualifies as AVG precisely when its evidence supports L3 or higher. Three trends stand out in this distribution. First, feedback adaptation is the most widely evidenced behavior: L3 is the modal level in nearly every domain, and a clear majority of the corpus meets or exceeds the L3 boundary at which a system qualifies as AVG. Second, evidence for the higher levels is unevenly distributed across domains: long-horizon recovery and cross-task improvement are concentrated in the image domain, appear less frequently in video, and become rare in the remaining domains, where reported systems cluster around single-loop correction rather than demonstrated long-horizon control. Third, fixed pipelines persist selectively rather than uniformly: they remain most visible in 3D/CAD, where procedural generation has an established workflow tradition, while other domains report few purely fixed systems. Cross-task improvement remains the least evidenced behavior overall, which motivates the training analysis in Section 11 and the frontier discussion in Section 13.

3. Analytical Components of Agentic Visual Generation

The preceding section defined agentic visual generation as state-dependent control over a visual creation trajectory. This section examines the components through which that control is organized during visual creation. A typical trajectory contains a task requirement, a selected operation, an artifact or execution result, an inspection or check, and a subsequent decision, among other task-relevant events. The decision may keep the current plan, change an input or tool, repair part of the artifact, replan, stop, or take another permitted control action. One controller may perform these steps, or different components may handle planning, execution, inspection, and related functions.

For cross-paper comparison, we use six descriptive components. Goal Understanding, Specification, and Planning cover the task goal, constraints, dependencies, and planning records that guide later operations. Memory covers information and records retained for access at later decision points. Tool covers callable models, editors, code environments, applications, simulators, and related interfaces, together with their input conditions and returned execution information. Perception covers signal acquisition, observation, interpretation, diagnosis, verification, and acceptance evidence. Action covers artifact transformation, information acquisition, execution, environment interaction, communication, coordination, trajectory control, recovery, stopping, and related operations. Cross-Task Self-Improvement covers how information from a completed trajectory is carried into a later task: episodic reuse retrieves earlier attempts or repairs, while cross-task self-improvement requires evidence that the update changes later behavior.

Each component corresponds to a part of the formal trajectory: Planning structures the goal g into commitments that later decisions condition on; Memory holds, in addressable form, the part of the accessible state $z_t=(x_t,s_t,h_t)$ that must survive across steps; Tool supplies the interface set ${\mathcal{I}}_t$; Perception computes the observation map $\mathsf{O}$ and returns the evidence $o_t$ on which the closed-loop criterion rests; Action applies the selected $a_t=(i_t,c_t)$ through the kernel $\mathsf{U}$; and Cross-Task Self-Improvement updates ${\pi }_{\theta }$ or extends ${\mathcal{I}}_t$ across trajectories.

Figure 9 shows the six components and their control dependencies, organized around the evolving visual artifact and the external task environment.

3.1. Six-Component Analytical Framework

3.1.1. Goal Understanding, Specification, and Planning

Component definition. This component interprets task information, expresses the intended visual outcome and its constraints, and organizes the remaining work into a plan. The input may combine a request, references, interaction context, domain requirements, and other task-relevant material at different levels of detail. The component turns this material into commitments that later operations can use, while keeping unresolved information attached to the part of the task it may affect. Its direct outputs include a current operational specification, unresolved items with their scope, and planned subgoals with their dependencies and order. When user information or task evidence changes a commitment or dependency, the affected specification and the remaining plan are updated along the same dependency structure.

A separate specification stage exists because natural-language requests compress intent through shared context: a sentence such as “make the character look older” fixes a target attribute while leaving composition, identity anchors, spatial relations, and rendering style to the generator’s defaults, and those defaults are resolved silently during sampling. An explicit specification externalizes the defaults as commitments. Each commitment names the intended property, its priority relative to competing properties, and the acceptance condition under which a rendered artifact satisfies it. Planning compiles the commitments and their dependencies into subgoals; Perception supplies the verification evidence that checks these acceptance conditions at runtime, and Action applies the resulting evidence to the affected part of the plan. This externalization supports two downstream mechanisms. First, localization: a rendered artifact that deviates from intent maps to the violated commitment and, through it, to a likely cause, so the repair targets the cause with the intent re-derivation already done. Second, granularity: repair operates on whatever unit the specification names, so the specification determines the vocabulary in which a failure can be described. The weight of this stage in visual creation follows from the cost structure of the domain: a wrong commitment detected at sampling time costs a full forward pass through a stochastic generator, and because identity, style, and composition are entangled in the rendering process, a late-detected commitment failure also damages the properties that were already correct.

Goal understanding. Goal understanding converts an utterance into a task target by separating three provenance classes: what the user stated, what the system inferred from context and references, and what remains unresolved. This split is the provenance record on which later revision depends. Natural-language requests systematically rely on shared background: the speaker leaves composition, style, or scope unstated because they expect the listener to fill them in. A controller that treats the prompt as complete projects its own unstated background into the specification, and the projection hardens into a commitment whose origin is invisible. When a later revision must preserve user commitments while repairing system-inferred ones, an unmarked provenance makes the distinction unavailable, and repair can overwrite a user-stated constraint while attempting to fix a system-inferred one. Unresolved material is therefore surfaced at this stage and assigned a resolution route: a clarification request acquires the missing information; a scoped assumption records a provisional commitment together with the condition under which it holds; a deferred item stays attached to the commitments it affects, so clarification or new evidence can locate it later. Each route keeps the unresolved material addressable, which is what allows incoming information to be applied at the correct layer of the specification.

From Idea to CAD uses interactive requirements elicitation to clarify design intent from sketches and text before construction [76]. Search Beyond What Can Be Taught studies generator-specific knowledge gaps and trains a reasoner to decide when external search is useful; SearchGen-Bench evaluates search-augmented generation, while the co-training experiments study coordination between the search reasoner and the generator [210]. Clarify Before Executing detects incomplete or ambiguous 3D requests and asks targeted questions before tool orchestration, progressively turning the clarified intent into an executable workflow [85].

Operational specification. An operational specification turns the interpreted goal into verifiable commitments, and its design covers two mechanisms: the structure of its commitments and the repair granularity it makes reachable.

· Commitment structure.

Each commitment records the intended property, its priority relative to competing properties, the condition under which it holds, and its provenance. The priority field resolves conflicts locally: when identity preservation and a local edit compete for the same region, the priority determines the winner without renegotiating the rest of the specification. The provenance field keeps user-supplied and system-inferred material distinguishable, so a later revision can preserve one class while revising the other. Commitments form a dependency structure: a change to a high-level commitment propagates to the affected subgoals downstream of it, while a local change touches only the connected part of the plan. This structure is what keeps specification updates tractable across a long trajectory.

· Repair granularity.

The reachable repair granularity is set jointly by the unit that the specification names and the unit that the artifact representation can address. Repair needs an addressable target: the specification supplies the address (which commitment failed and where), and the representation supplies the storage (what structure exists to receive the change). An object-level specification over a layered representation supports object-level repair; a prompt-level specification over a pixel representation supports whole-output regeneration, because the representation exposes no element that a repair could address even when the specification names one. Specification granularity therefore has to be chosen with the representation in view, and when repair precision matters the two are designed together.

NEWTON identifies a specification bottleneck in physics-grounded video generation: the physical specification must provide sufficient, dynamically relevant, and verifiable information for the simulator and generator to realize the intended behavior [7]. Divide and Conquer makes the objects, attributes, relations, and scene layout of a compositional request explicit before composing the image [8]. MetaPoint converts a free-form image-editing request into structured object-level commands, including precise spatial information for the requested edit [211].

Plan construction. Planning compiles the specification into a dependency graph of subgoals, and its design covers three mechanisms: plan representation, decomposition depth, and the balance between alternatives and commitments.

· Dependency graph.

Each node carries the subgoal it addresses, its preconditions, the action intended to satisfy it, and the post-conditions that signal completion; edges record the order and the information dependencies among nodes. The plan may be written in natural language, as a program-like structure, as a tree, or as a hierarchy; the format matters less than the properties it exposes: order, dependency, and alternative continuation. These are what later decisions condition on.

· Decomposition depth.

Granularity is the central design variable, and hierarchical planning theory makes the trade-off precise: a plan is a decomposition in which compound tasks are refined into subtasks until primitive, executable ones remain, and the chosen depth determines how much work is decided before execution and how much is deferred to it. Visual generation sharpens this trade-off because intermediate artifacts are samples from a distribution. A plan that names a concrete intermediate appearance commits to one draw from that distribution and is invalidated when a different draw arrives; a plan that references dependencies, post-conditions, and acceptance conditions survives across draws, because those invariants hold regardless of which sample the generator returns. Robust visual plans therefore record alternative tools at each node and re-derivable lower-level decompositions, leaving leaf-level decisions to execution time. This is the contingent-planning pattern from partially observable domains, adapted to a setting where the observation step is a verification check performed by Perception on a rendered artifact.

· Alternatives and commitments.

Alternatives carry a cost that scales with their number: each recorded alternative enlarges the search space at execution time and the bookkeeping at plan time, while each additional commitment shrinks the space and forfeits adaptivity. The right operating point follows from the generator’s variance and the cost of verification: a high-variance generator with cheap verification rewards alternatives; a low-variance generator with expensive verification rewards commitments.

GenArtist represents generation and editing operations in a planning tree, with sibling nodes encoding alternative tools for the same action [23]; it operates at the alternatives-heavy end of the spectrum and pays with a larger search space. LightVA recursively decomposes an analytical request into data and visualization tasks for subsequent execution [88]. From Plans to Pixels generates structured atomic decompositions for open-ended, long-horizon image-editing requests and conditions tool and region selection on the resulting plan [212]; it commits to one decomposition to make region-level tool selection tractable, giving up the alternative continuations that GenArtist preserves. Each occupies a defensible operating point given its generator variance and verification cost.

Plan revision. Planning remains revisable when new information changes a commitment, a dependency, or a condition governing the next action, and its design covers two mechanisms: the revision procedure and staleness detection.

· Revision procedure.

Revision theory distinguishes full replanning, which re-derives the plan from the current state, from plan repair, which applies a local transformation to the existing plan while preserving its valid parts. Repair is the cheaper option whenever the dependency structure can identify the subgoals downstream of the change; replanning is the fallback when the change invalidates the plan’s top-level structure. The procedure follows the dependency graph: classify the incoming information as a refinement, a violation, or an addition to an existing commitment; locate the affected commitments; compute the affected subgoals (the downstream closure of the changed commitment); and re-derive only those subgoals, retaining the rest. The revision scope therefore equals the downstream closure of the changed commitment, which keeps the relationship between new information and the resulting planning decision explicit.

· Staleness detection.

A recurring failure is the stale plan. A plan is valid under the artifact state that held at construction time; execution mutates the artifact, so any subgoal whose preconditions referenced a mutated state becomes stale without an explicit invalidation event. The mechanism-level guard binds subgoal applicability to post-conditions evaluated at decision time: the precondition check itself detects staleness, and the damage stays local to the affected subgraph. When the incoming information is task evidence, such as a failed verification or an unexpected artifact, the revision is evidence-driven. When it is user information, the revision is preference-driven. Both follow the same procedure, and the distinction lies in who owns the resulting commitment.

Adaptive Task Reformulation changes the task representation and selects a new reformulation route when direct image editing fails [213]. IMAGAgent orders constraint-dependent editing sub-tasks and uses reflection on intermediate results to trigger self-correction or retry [214]. MIRA predicts one atomic editing action from the current visual state, feeds the updated image back into the next decision, and includes a stop decision when the instruction is satisfied [19].

Taken together, these four operations form a progression: task interpretation produces provenance-marked commitments, commitments compile into a dependency-aware plan, and incoming information updates the affected subgoals of the changed commitment. The resulting records provide the planning substrate that later operations condition on, while keeping unresolved assumptions and plan changes traceable to the information that produced them.

A representative implementation of this planning pattern is shown in Figure 10. GenArtist decomposes a complex request, organizes candidate operations in a planning tree, and routes the selected steps to generation and editing tools with intermediate verification [23].

3.1.2. Memory

Memory definition. Memory is information retained from an interaction so that a later decision can use it with the current task information. A record becomes memory when the system selects it from an interaction, gives it a readable and addressable representation, and makes it available at a later decision point. The retained information can describe a requirement, reference, visual observation, artifact version, action outcome, or prior experience. Its representation can be textual, visual, structured, executable, or multimodal, with links that preserve the relation between a record and the operation or result from which it came. Interaction history and current context provide source information; memory is the selected record that remains available for later use. In the formalization of Section 2.2, Memory realizes the parts of the accessible state $z_t=(x_t,s_t,h_t)$ that must survive across steps in an addressable form: it carries selected content from the write at one step to the read at the next, and because the observation map $\mathsf{O}$ reads from $h_t$ alongside $x_t$ and $s_t$, a retrieved record enters the next decision through the same channel as direct inspection of the artifact.

The design pressure that makes retention a selection problem is context budget. A visual trajectory produces frames, intermediate renders, tool traces, and verification outputs whose volume exceeds what a decision context can carry across turns. Retention is therefore governed by expected decision value: candidate records compete for storage and for injection cost at later decisions, and the records that survive are those whose expected contribution to a later decision justifies both costs.

Qwen-Image-Agent treats memory as a context-grounding source for historical and personalized information during image generation and editing [215]. The Cognitive-structured Multimodal Agent externalizes visual information into episodic visual memory and uses a retrieval engine to reactivate relevant visual episodes across turns [131]. LAVE maintains a memory buffer of prior conversations and combines recent history with the current instruction when constructing a video-editing action [59]. These systems show the same basic operation at different scopes: information produced earlier is selected and supplied to a later generation decision.

Memory formation, update, compression, deletion, and retrieval. Memory passes through five operations: formation, update, compression, deletion, and retrieval. These operations jointly determine what survives a trajectory and what reaches a later decision.

· Formation.

Memory formation selects information with potential future utility from a request, reference, action, artifact output, observation, feedback signal, or user correction. The selection may preserve an event, extract a proposition from it, summarize several related events, or link an editable source to the result it produced. The selection criterion covers the information’s expected value for a later decision, together with its scope, reliability, and provenance. During a multi-turn image edit, the system retains a user-confirmed subject identity and a region that has already met the task constraints, and marks an unconfirmed stylistic preference as unresolved. During long-video creation, frame-level observations are collected as candidates and converted into a record about an entity or relation that persists across the relevant part of the trajectory.

· Update.

Updating incorporates new information into an existing record or set of records. It can append a new event, revise a value, attach evidence, change the validity interval, merge compatible entries, or preserve competing versions when the evidence is unresolved. Each update records the scope, source, and validity interval of the revised content, which makes staleness detectable at retrieval time, where the expired record is filtered before it reaches a decision. When a later edit changes the approved appearance of an object, the corresponding memory points to the new version while retaining the earlier version as provenance. When a new observation conflicts with an earlier one, the record holds both claims together with their scope conditions, and verification evidence from Perception closes the conflict at a later decision point.

· Compression.

Compression reduces the amount of stored information or the context supplied to a decision while preserving the content needed for that decision. Its operations include summarization, which replaces repeated observations with a stable description; deduplication, which merges records that express the same fact; and hierarchical storage, which links a compact summary to more detailed evidence. The two record classes divide the work: descriptive records are cheap to store and retrieve and support reasoning, while raw visual records carry appearance at higher storage and injection cost, and a description alone lacks the information needed to render the described content again. Hierarchical storage therefore links the description to its supporting frames, so reasoning steps retrieve the description and appearance-critical steps reach the frames through the link; a long sequence of consistent character observations is represented by a compact identity record with links to the frames that support it, and an exceptional frame remains available for diagnosis. The compressed record preserves the constraints, dependencies, uncertainty, and provenance required to recover or revise the relevant part of the artifact.

· Deletion.

Deletion removes a record, marks it inactive, or lets it expire when its validity, scope, utility, safety, or maintenance cost no longer supports retention. Local deletion removes an obsolete tool parameter or a failed intermediate version while the task requirement and the last verified artifact remain available. Conditional deletion retains a record for provenance while excluding it from ordinary retrieval after its validity interval ends, so the record stays available for audit and leaves the decision surface. Expiry based on the validity interval gives time-driven deletion without explicit removal.

· Retrieval.

Retrieval makes retained information available to a current decision. The query is constructed from the task specification, the current observation $o_t$, and the information needed for the next operation. Retrieval proceeds in stages: candidate records are matched on semantic distance, hard filters on scope, identity, and validity remove the records whose scope or validity interval fails the current decision, and post-retrieval processing orders, summarizes, and binds the remaining records to the current artifact. When a later shot is edited, retrieval returns the latest approved identity record, the relevant reference frame, and the prior repair that affected the same dependency, and unrelated experiments stay outside the decision context. Retrieval occurs at task initialization, at selected turns, or when a new observation creates a need for earlier evidence. Its role is complete when the returned record is available to the subsequent decision, such as choosing an input, editing a source representation, selecting a tool, revising a plan, or stopping. It does so as part of the observation $o_t$, on the same footing as evidence acquired directly from the artifact or the environment.

COMFYCLAW distills trajectories, execution errors, and verification feedback from image-workflow construction into reusable Agent Skills and progressively discloses those skills when later workflows require them [45]. DataEvolver converts rejected text-rich image samples and their verification causes into semantic feedback, retains useful feedback entries, merges near-duplicate observations, and uses the resulting experience memory to revise later retrieval queries and generation prompts [216]. VideoWeaver stores process and output evidence from long-video generation cases and uses the accumulated context to summarize category-level patterns and evolve reusable composition skills [70]. These systems instantiate selected parts of formation, updating, compression, retention, and retrieval at different scopes.

Memory types. Memory types describe different properties of a retained record along three dimensions.

· Temporal dimension.

Short-term memory supports decisions within a turn or the current trajectory; long-term memory remains available across turns, scenes, or tasks. Both roles can be implemented by the same store when the formation and retrieval schedules attached to their records differ: a type is a property of the schedule attached to a record.

· Functional dimension.

Working memory holds the active task workspace; factual memory holds relatively stable information about users, environments, references, or visual entities; episodic memory holds particular attempts and outcomes; procedural or skill memory holds reusable action patterns. These four roles map onto the three functions described below: working and factual records carry task continuity, episodic records support episodic reuse, and procedural records support procedural reuse.

· Representation dimension.

Textual or token-level summaries preserve requirements and prior decisions; visual or multimodal records preserve appearance, identity, and visual evidence; structured records and graphs expose entities and dependencies; executable or artifact-linked records connect source code, workflows, or other editable representations to their outputs. The carrier decision follows from a property of visual content: identity is carried by appearance, a textual proxy preserves the decisions about a character, and the appearance that makes the character recognizable lives in the visual record, so the two classes are combined through links when both reasoning and appearance matter. Access can be private to one role or shared among several roles: private records carry role-local context, shared records implement the coordination surface among roles, and the shared partition carries per-record ownership and an update policy. The information channel can be textual, visual, structured, executable, or multimodal.

DreamFactory explicitly separates short-term information within a scene from long-term information that must persist across scene transitions, extracting a Base Description containing style, background, and character attributes and combining it with preceding-frame context for later key-frame generation [56]. ViMax uses global narrative context, retrieval-augmented generation, and graph-based cross-shot dependencies to carry story information into local video-generation decisions [217]. ShareVerse implements spatiotemporal memory retrieval and conditions later video frames on retrieved memory frames to preserve a shared environment across asynchronous agent trajectories [218]. Together, these examples cover temporal, functional, representational, and shared-access dimensions.

Functions in agentic visual generation. Memory in agentic visual generation serves three core functions.

· Task continuity.

Task continuity preserves requirements, references, identities, dependencies, artifact versions, and decision evidence that remain relevant as a visual creation trajectory proceeds. A later operation continues from an established task interpretation, inspects an earlier source or artifact version, preserves completed work, and revises the unresolved part of the trajectory. In a multi-turn image edit, a retained subject record and the latest verified image support a new local edit while the decisions established in earlier turns remain in force. In long-video creation, retained entity and scene records carry an established relation into a later segment while local content changes.

· Episodic reuse.

Episodic reuse retrieves a record of a particular earlier attempt, outcome, failure, or repair for a new but related request. The record supports the new request when its task conditions, selected actions, resulting artifact, and review evidence remain interpretable. A system retrieves a prior trajectory for a related visual defect, uses the successful intervention as an initial candidate, and adapts it to the current artifact and constraints; it also retrieves a failed episode to avoid repeating an action that produced the same conflict, or to select the observation that distinguished the earlier failure from a successful case. Consistent with the boundary fixed in the section introduction, the retrieval itself belongs to Memory, and the persistent adaptation that turns retrieved experience into a demonstrated later-task behavior change belongs to Cross-Task Self-Improvement.

· Procedural reuse.

Procedural reuse stores a reusable action pattern distilled from one or more episodes. The pattern can specify an operation order, preconditions, parameter relations, expected observations, or a repair strategy, while task-specific values are supplied by the current request. The form of retention determines what the pattern can do: retained as guidance, it informs how the next action is constructed; registered through tool creation as a callable capability with declared inputs and execution conditions, it enters the interface set ${\mathcal{I}}_t$ and extends the actions available to later decisions. A reusable visual-document skill, for example, preserves the sequence of inspecting source structure, rendering the result, checking the page, and revising the responsible source; a video-composition skill preserves continuity checks while the current scenes and assets change.

Agent Banana uses Context Folding to abstract a growing editing history into asset-, execution-, and planning-level records; its persistent ActionContext keeps the verified effective editing path and the associated image states for later turns, providing task continuity across successive edits [219]. Generation Navigator represents each subsequent action using the original prompt, prior actions, generated images, reviewer feedback, and scores accumulated along the trajectory, keeping task-local evidence available for the next decision [29]. SEAR builds a self-evolving episodic memory that consolidates high-reward restoration trajectories into retrievable expertise and uses a state fingerprint to select relevant prior trajectories, supporting episodic and procedural reuse across related restoration requests [220].

The interaction between short-term trajectory records, evaluation feedback, self-reflection, and longer-term experience is illustrated in Figure 11. Reflexion places these memory forms inside an iterative loop in which feedback from one trial is converted into information used by later trials [209].

3.1.3. Tool

Tool definition. This component represents a visual-creation operation as a callable tool and records the result of invoking it. The input to a tool call is constructed from the current task specification, the available task state, and the operation selected for the current subgoal. The tool boundary determines which inputs can be supplied, which artifact or environment is affected, and which outputs are returned to the ongoing trajectory. A returned result can contain a new or modified artifact, a structured state, an execution status, an error, or other information exposed by the executor. The interface may also expose preconditions, side effects, latency, cost, or reversibility, among other interface properties, when the implementation defines them. In the formalization of Section 2.2, Tool instantiates the interface set ${\mathcal{I}}_t$, and tool creation is the operation by which a new capability enters that set. A tool is a bounded, callable capability that carries out an operation on a visual artifact, an editable representation, or a task environment. Its design covers three mechanisms: the call and return contract, the addressability of results, and the execution properties that visual generators add to the contract.

· Call and return.

A tool’s definition connects an operation name or endpoint to an input representation, an execution procedure, and a return representation. The input can be a prompt, image, mask, structured record, program, parameter set, or another task-specific value. The output can be a rendered result, an edited source, a geometric object, a chart, a document, an interface state, a simulator trace, an execution message, or another task-specific result. For example, an image-editing tool may accept a source image, a mask, and an edit instruction, invoke an editor, and return an edited image with the version or file identifier needed for the next call. A CAD tool may accept a parametric program and return geometry together with compiler or kernel status.

· Addressable results.

A useful interface keeps the relationship between the supplied input and the returned result addressable, so a later operation can identify the artifact version or environment state produced by the call. The interface can expose validation rules and failure states at the boundary, while the interpretation of whether a result satisfies the task remains a separate operation.

· Stochastic and non-atomic execution.

Visual generation adds two execution properties to this contract. First, the generation tools behind visual creation are stochastic. The same input to a diffusion-based generator can return different artifacts across calls, and even a fixed seed does not guarantee identical output across hardware, precision, or serving configurations. This has three direct consequences for the interface contract. A retry under the same condition draws a new artifact and does not recover the failed one. Verification is stated over the output distribution, and a single artifact is one sample against it. The interface returns the artifact itself, because the controller needs the concrete version to decide the next step. Second, tool calls can be non-atomic in real execution environments. A call may time out after the underlying operation has partially taken effect. This makes idempotency and version identification part of the interface contract. Repeating the call should have the same intended effect on the recorded state as a single call, and version identification binds each recorded state to the specific draw that produced it.

GenClaw provides an example of code-driven image operations in which a visual instruction is translated into executable operations over a canvas representation [221]. TOOLCAD formulates text-to-CAD interaction around tool calls to a CAD engine and uses trajectories containing tool execution outcomes [83]. CAD-Assistant connects a vision–language model to executable CAD functions so that language decisions produce CAD operations and returned model states [78]. These systems expose different interfaces, while each makes the operation boundary part of the available execution space. GenClaw and CAD-Assistant sit at two ends of the boundary spectrum. The former wraps pixel-level operations and carries their stochasticity at the interface. The latter wraps deterministic kernel calls, where reproducibility is available and geometric validation replaces distributional verification.

Tool types. Tool types describe the operation supplied by an interface and the substrate in which it executes. The classification covers two mechanisms: the function groups an interface can join, and the cost profile that the group separation produces.

· Function groups.

For agentic visual generation, the working groups include visual generation and editing tools; visual perception and analysis tools; retrieval and external-information tools; code, application, and rendering execution tools; and simulation or task-environment tools, among related tool functions. The groups describe the function provided to the current trajectory. One interface can combine several functions, such as an application tool that accepts editable code, renders a visual artifact, and returns execution messages. A perception or analysis tool contributes an output such as a measurement, label, relation, data summary, or another task-relevant signal. That output can inform a later decision when it is related to the current task requirements and available state.

· The five groups.

Visual generation and editing tools produce or modify the artifact used by the task; a call may map a prompt and reference image to a new image, or a source frame and edit region to a revised frame. Visual perception and analysis tools expose properties of an artifact or its source; examples include returning detected objects, text, measurements, or a data summary. Retrieval tools supply references, examples, or external information that can be used to construct later inputs. Code, application, rendering, and application tools execute programs or operate within an application environment; a plotting tool can turn a table and Python code into a figure, while a presentation tool can turn source code into rendered pages. Simulation tools execute a candidate representation under a modeled environment and return a trace or status that can be passed to a later decision; a physics tool can receive an event program and return simulated positions, contacts, or an execution failure.

AMACE uses separate agents for chart-code generation, chart rendering, and chart-quality assessment [98]. PlotGen combines code generation with numeric, textual, and visual feedback agents for scientific visualization [99]. METAL separates chart generation, visual critique, code critique, and revision in a chart-to-code workflow [222]. These examples show how tool type is determined by the operation and execution substrate, while the same system can expose several related interfaces.

Tool creation. Tool creation constructs a callable capability that can be registered and invoked by a later decision. Its design covers three mechanisms: the construction routes, the timing and scope of registration, and the composition of primitives.

· Construction routes.

The construction process may generate an implementation, wrap an existing model or application programming interface, compose several primitive operations, or attach an input–output interface to an executable procedure, among other constructions. It receives the intended operation, its dependencies, and the values that must cross the interface. It produces a callable object together with the information required to invoke it, such as accepted arguments, return format, environment requirements, and reported execution states, together with related interface metadata. Tests, example calls, or other checks can establish that the new interface is executable before it is placed in the available action space. A generated image, a source program, a workflow description, or a memory record can contribute to this process when the system uses it to implement, parameterize, document, or test a callable operation.

· Timing and scope.

Tool creation can occur before a task, when a system prepares a library of domain operations, or during a task, when an unavailable operation is assembled from accessible components. For example, a system can wrap a renderer as a callable function that accepts source code and asset paths, runs the renderer in a specified environment, and returns page images plus an execution status. A newly created tool can be specialized to the current artifact, generalized for later requests, or composed into a higher-level operation, among other scopes. Its utility depends on the compatibility between its input and output representations and the state of the execution environment. The record preserves the implementation version, dependencies, invocation conditions, and returned status needed to reproduce a later call. Tool creation supplies an additional callable capability; selecting when to use it and interpreting its result are separate decisions.

· Composition and localization.

Composition combines primitive operations behind one interface. A composed tool hides the internals of its primitives from the controller, which reduces interface complexity but also removes the localization information a later repair would need when the composed call fails midway. The composed interface must therefore record this provenance explicitly, noting which primitive produced which intermediate result and with what inputs, to preserve repairability when composition depth makes internals opaque.

Tool selection and routing. Selection identifies the capability for the current subgoal, and routing fixes the path the task travels through that capability. Both are established from the task and revisable as runtime evidence changes the requirements for the next operation.

· Selection.

Tool selection identifies the capability used for the current subgoal. The selected capability may be a generator, editor, analysis function, retrieval service, application, simulator, or another available executor. Selection uses the task requirements, current artifact and state, interface compatibility, and operating constraints such as quality, cost, latency, or side effects. Its output is a tool choice together with the arguments and conditions needed for invocation. Selection quality is bounded by two orthogonal limitations. An incomplete ${\mathcal{I}}_t$ leaves the correct capability unavailable regardless of the selection policy. An underspecified interface description leaves the correct capability present while its contract goes unrecognized. Tool creation addresses the first limitation, and interface documentation addresses the second. An interface description records the accepted input representation, the return representation, and the reported execution states in machine-processable form, which lets the selection policy match a subgoal to a capability from its declared contract.

· Routing.

Routing describes how the task or intermediate result is sent through the selected capability or a sequence of capabilities. A route can connect one tool directly to the next operation, pass an artifact through several compatible tools, or direct different subgoals to specialized paths. Selection therefore answers which capability is used. Routing describes the path, ordering, and handoff through which that capability participates in the task. Both can be established from the initial request and can be revised when an artifact, execution status, error, or other runtime information changes the requirements for the next operation.

Tool use across artifact, execution, and coordination settings. Tool use is organized along three dimensions: the artifact being produced or edited, the execution substrate, and the organization of tool calls.

· Handoff.

Tool use is the invocation of a selected capability and the handoff of its returned result to the next operation. The handoff preserves the representation needed by the recipient: an image or video render can be passed as a visual artifact, source code as an editable representation, a chart table as structured data, a CAD program as executable input, a simulator trace as an environment result, or another task-specific representation. The return record can include success or failure status, changed files or objects, generated artifacts, execution metadata, or related execution information exposed by the interface. A later operation can then continue from the returned state, supply a revised input, select another compatible tool, or deliver the artifact to the user.

· Artifact dimension.

The first dimension is the artifact being produced or edited. Pixel and temporal artifacts include images, masks, reference frames, clips, and scene segments. Structured artifacts include source code, layer hierarchies, scene descriptions, geometric programs, data mappings, layouts, and interface states. A portrait-editing call may combine a source image, a subject mask, and an edit instruction, then return a new image and an editable region for another local change. A CAD call may pass a parametric program and return geometry with compiler or kernel status. A chart, presentation, or Web call may pass data and source code and return a figure, rendered pages, or an updated interface state.

· Execution substrate.

The second dimension is the execution substrate. A selected tool can call a model or API, run code in a runtime or sandbox, operate a desktop or Web application, invoke a renderer, query an external information service, or execute a simulator or other task environment. A code-mediated operation can run a plotting script and return a figure plus execution status. An application-mediated operation can modify a slide or interface and return rendered pages or an updated state. A simulator-mediated operation can receive an event program and return a trace, contact states, or an execution failure. The handoff records the input representation, the executor, and the returned artifact or status so the next operation can address the correct file, object, or environment state.

· Call organization.

The third dimension is the organization of tool calls. One decision process can construct inputs and invoke tools directly. A staged workflow can pass the output of one operation to a later operation with a different interface. A multi-agent workflow can assign generation, editing, programming, rendering, or integration calls to different roles. Role decomposition purchases specialization at the price of the handoff. Each role boundary re-encodes the representation and ends the context the sending role treats as shared. The execution record therefore states what travels with the payload: the input, the acting role and tool, the returned artifact or status, and the decisions and constraints behind the artifact. Mora organizes model and agent roles for text-to-video, image-to-video, and editing workflows [223]. GenMAC distributes compositional text-to-video work among collaborating agents [65]. In such workflows, a prompt interpretation can be handed to a generation role, a generated result can be handed to an editing or integration role, and a rendered output can be handed to a later delivery step. The execution record retains the input, the acting role and tool, and the returned artifact or status so that each handoff remains traceable.

· Combined trajectories.

These dimensions can be combined in the same trajectory. A task may begin with a model call over a reference image, move to code execution for a structured layout, pass the result to an application renderer, and then use a simulator or interaction environment for a task-specific check. Across these combinations, the tool-use record identifies the operation invoked, the representation crossing the interface, the executor or role that acted, and the result made available for the next decision.

Figure 12 shows a complementary view of tool use at the interface level. Toolformer samples candidate API calls, executes them, filters them using their contribution to prediction, and incorporates the retained calls into an augmented dataset [224].

3.1.4. Perception

Perception definition. This component obtains and interprets information about a visual artifact, its editable representation, its execution process, the task environment, references, and interaction. Its input is a task-relevant signal source; its processing acquires, extracts, compares, localizes, or checks information; and its output is structured evidence that a later Action can use. This evidence may contain a score, a pass or graded status, a failed requirement, a location, a likely cause, uncertainty, supporting observation, or a candidate target. The mechanism can be instantiated for different visual artifacts and execution settings.

Perception types and roles. Perception types and roles covers the kinds of signals it operates on, and the role is fixed by its position in the perception-action loop: Perception supplies the comparator’s evidence and Action applies the correction, so evidence production and correction execution are two stages of the same feedback loop.

Perception can operate on several kinds of information. Appearance perception concerns visible content, such as object presence, text rendering, color, texture, and layout. Structural perception concerns the representation that produced the visible result, such as source code, layers, scene relations, geometry, data mappings, or interface structure. Temporal and relational perception concerns changes across frames, shots, turns, entities, or dependent elements. Execution and environment perception concerns compiler messages, runtime status, application state, browser state, simulator traces, or other signals returned by an external process. Reference and user perception concerns comparisons with a supplied image, specification, example, preference, or correction. For example, in a slide workflow, the trajectory may combine page appearance, source structure, and parser status; a CAD workflow may combine rendered geometry, kernel measurements, and program state; a video workflow may combine sampled frames with identity and temporal relations. This type-based decomposition lets evidence be collected and combined per decision target, so that each check receives the signal class suited to its question and a coverage gap in one channel is covered by another.

Signal acquisition and observation. Signal acquisition and observation covers the selection of sensor channels and the trade-off among their coverage, timing, localization, and cost.

Signal acquisition obtains information from the artifact and its execution context. A system may, for instance, sample outputs at a fixed cadence, check a known risk point, select a crop or frame range, inspect source code or geometry, query a data record, retrieve a reference, or read a compiler, renderer, browser, application, or simulator result. The useful properties of a channel include its coverage, timing, localization, and cost. For instance, a scheduled render check may follow a scene transition, while a local inspection may be selected when uncertainty is concentrated in one region. These properties shape which signals are collected and when they become available for interpretation. The selection follows the active-perception principle: each acquisition carries a cost, and information gain depends on where the current uncertainty concentrates, so channel choice optimizes information value against acquisition cost. Scheduling cheap checks before expensive tests is the same optimization applied to the ordering of checks within a single verification pass.

In a program-mediated video workflow, Kubrick inspects Blender renders within its synthetic-video process [9]. The artifact and execution substrate determine which signals become available for interpretation.

Interpretation and diagnosis. Interpretation and diagnosis covers the mapping from an acquired signal to a task requirement, a possible violation, and an affected part of the trajectory.

Interpretation connects an acquired signal to a task requirement, a possible violation, and an affected part of the trajectory. Diagnosis can identify a region, frame, object, component, or operation; associate the observation with a likely source; and retain competing hypotheses or confidence when the system reports them. The usefulness of a diagnosis depends on how directly it connects the observed signal to a later action target. Diagnosis proceeds by abductive reasoning: it generates the best-explanation hypothesis for the observed symptom, and when symptoms do not uniquely determine a hypothesis, the system retains competing hypotheses and selects the next check with the highest information gain. Root-cause localization accuracy directly determines the direction and speed of remediation: an incorrect judgment directs repair resources at an unrelated part, while an accurate one targets the violated commitment directly.

Concrete cases include CADSmith, which combines execution checks, CAD-kernel measurements, and visual assessment to separate runtime, dimensional, and morphological problems [84]; IterCAD, which uses executable sandbox feedback and geometric evidence in interactive CAD generation and editing [80]; PhysAgent, which uses stage-specific simulation results to distinguish reconstruction failures from dynamics failures [67]; and DirectorBench, which organizes long-form video evaluation around checkpoint-level specifications and profile-aware bottleneck reports [225]. These cases treat diagnosis as an interpretation of available evidence; the repair, retry, or route change remains a subsequent Action.

Verification and acceptance evidence. Verification and acceptance evidence covers the checking of acceptance conditions and the production of the corresponding evidence.

Verification checks an explicit requirement, acceptance condition, or property of the current result. Relevant targets include perceptual and semantic correspondence; relational, spatial, and geometric validity; temporal continuity and identity consistency; data, factual, and numerical fidelity; executable behavior and structural validity; physical plausibility; and human acceptance when a human review is part of the reported system. For a text-rendering requirement, this may involve OCR; for a CAD constraint, a geometry or compiler check; for a physical event, a simulator; and for an open-ended visual property, a learned judge or human review. Different targets require different evidence, and each check exposes a particular uncertainty.

A verification process can combine exact tests, reference-based comparisons, learned judges, source-level checks, simulations, and human review. It can record coverage, granularity, latency, confidence, and disagreement, and can schedule cheap checks before expensive tests. These checks answer different questions about the current result and can be combined when the artifact exposes multiple representations. Verification faces the oracle problem: a test oracle is a predicate that decides whether a behavior is acceptable, and for properties with an explicit specification an exact check serves as that oracle. For open-ended visual properties such as aesthetics or style consistency, no ideal oracle exists, so a learned judge or human review supplies an approximate oracle whose confidence and disagreement are recorded for Action to weigh at decision time.

Figure 13 illustrates the observation context used for visually grounded agent evaluation. It combines web resources, page content, task-specific visual references, and an agent action interface, making the perceptual inputs available to subsequent decisions explicit [226].

3.1.5. Action

Action definition. Action is an operation selected at a decision point that changes a visual artifact, editable representation, external environment, available information, interaction history, or control trajectory. It receives the current goal and specification, Memory records, artifact and environment state, available Tool interfaces, and Perception evidence. It constructs an input, invokes an operation, and incorporates the returned artifact, state, status, communication result, or control update into the running trajectory. An action record can therefore identify the selected operation, its input and target, its executor or role when reported, its returned result, and available cost, latency, reversibility, or side-effect information. The action space includes artifact transformation, information acquisition, execution, environment interaction, communication, coordination, and trajectory control, among related operations.

· Operating arrangements.

The same action principle applies across several operating arrangements. A single decision process may invoke a model or editor directly; a staged workflow may pass a source representation through generation, rendering, and inspection; and multiple roles may divide generation, editing, programming, integration, or delivery. For instance, a reference image can be passed to a generator, the generated representation to a code or application runtime, and the rendered result to a later operation.

Artifact actions. Artifact actions covers the creation or modification of a visual artifact or its editable representation, and the two mechanisms it engages are the input-to-artifact mapping and the granularity at which the operation preserves or destroys completed work.

Artifact actions create or modify a visual artifact or its editable representation. The mechanism maps a task-specific input to a changed artifact while preserving the representation needed for a later operation. It can modify pixels, frames, masks, source code, layers, geometric programs, data mappings, layouts, or executable interfaces. A prompt and reference image can produce an image; a source frame, region, and edit instruction can produce a revised frame; a parametric CAD program can produce geometry and execution status; and data with source code can produce a figure, transformed data, or rendered pages. The operation may be global or localized, and its granularity determines which completed properties remain available for later steps. A destructive operation overwrites the intermediate state and leaves the overwritten content unrecoverable, so a later repair that targets an overwritten region must regenerate it. A non-destructive operation keeps the prior state addressable through a parameter record or a versioned layer, so a later step can roll back to it. A local edit touches only the selected region and preserves the completed properties outside it, while a global regeneration redraws the whole artifact and risks the properties that were already correct.

T2I-Copilot combines structured prompt interpretation, model routing, segmentation, interactive canvas operations, and quality evaluation in a text-to-image workflow [32]. AutoStudio separates prompt extraction, subject management, layout construction, layout supervision, and drawing for multi-turn image generation [26]. Anywhere assigns distinct operations to foreground understanding, diversity enhancement, object-integrity protection, and text consistency in foreground-conditioned image generation [227]. The examples differ in artifact and operation, while each makes an artifact change available to later steps.

Information acquisition actions. Information acquisition actions covers the obtaining of material for a later operation, and the mechanism is the construction, dispatch, and return of a query or inspection request.

Information acquisition actions obtain material for a later operation. The mechanism constructs a query or inspection request, sends it to a source, and returns a reference, measurement, source fragment, preview, execution message, or other task-relevant record. It includes search and retrieval of references or external knowledge, data queries, source-code and file inspection, browser or application inspection, environment-state inspection, rendering of crops or previews, selection of a frame range, invocation of a perception or analysis service, and read or write operations on retained task records. For instance, a system may retrieve a visual reference before generation, inspect a source file before editing, or render a local crop before selecting a region-level action. Perception interprets the returned material; Action performs the operation that obtains it.

Execution and environment actions. Execution and environment actions covers the execution of an operation through a tool boundary, and the mechanism is the translation of a selected operation into an executable call that produces a state change.

Execution and environment actions operate through a model or API, a code runtime or sandbox, an application or browser, a renderer or compiler, a simulator, or another task environment. The mechanism translates a selected operation into an executable call, applies it to a target artifact or environment, and returns the resulting artifact, state, status, error, or trace. A CAD action can submit a program to an executable engine, an application action can modify a source file and render a page, and a simulator action can execute an event program and return a trace or failure. The Tool section defines the callable boundary; Action concerns the operation performed through that boundary and the state change it produces. When the operation produces side effects that are difficult to reverse, the execution environment can preserve a snapshot or checkpoint of the pre-action state, so a failed or undesired action can be rolled back to that state and retried with a revised input.

Communication and coordination actions. Communication and coordination actions covers the movement of information between roles or authority boundaries, and the mechanism is the selective forwarding of a payload across the boundary.

Communication and coordination actions move information between roles or authority boundaries. A sending operation selects what information to transmit, formats it for the recipient, and records the response or resulting handoff. The action can involve delegation, message passing, synchronization, aggregation, role handoff, delivery of an artifact or structured state, clarification requests, approval requests, rejection handling, progress reporting, or human override. In a multi-agent workflow, a prompt interpretation can be sent to a generation role, a generated result can be sent to an editing or integration role, and a rendered output can be sent to a later delivery step. Mora organizes model and agent roles for text-to-video, image-to-video, and editing workflows; GenMAC distributes compositional text-to-video work among collaborating agents [65,223]. Because the channel between roles has limited bandwidth, the sending operation compresses or selects the payload, and the selection determines what the recipient can condition on; the evidence dropped at the boundary leaves no trace in the recipient’s context, so the recipient cannot distinguish a silent exclusion from an absent signal.

Evidence-conditioned action and trajectory control. Evidence-conditioned action and trajectory control covers the use of Perception output to determine how the trajectory continues, and the mechanism is the mapping of evidence to an admissible response that drives the next state.

Evidence-conditioned action uses Perception output to determine how the trajectory continues. The mechanism receives a score, failed requirement, location, likely cause, or uncertainty, maps it to an admissible response, and performs that response. A critique can be sent to a prompt or code generator, a failed requirement can be applied to a local edit or revised input, an execution error can be passed to a retry or routing decision, and a result can be communicated to another role or user. A verified state can support continuation; an unresolved state can lead to more evidence, clarification, recovery, or stopping. The mechanism instantiates the belief-action coupling: Perception updates the agent’s belief about the current state, and the policy selects the action with the highest expected value under that belief, so the verification verdict and its confidence enter the action choice through the same channel as the task specification.

· Trajectory-control actions.

Trajectory-control actions include tool, model, or role selection and routing; prompt, input, plan, dependency, or parameter updates; retry, branching, continuation, or path substitution; local repair, replanning, rollback, and bounded relaxation; stopping, termination, or deferral; and human authorization, override, or escalation. CoSTA* formulates multi-turn image editing as cost-sensitive tool-path selection [10]. FaSTA* separates fast and slow tool paths and mines reusable subroutines for later editing decisions [228]. These systems treat trajectory control as cost-sensitive search over the action space, where each candidate action carries an expected cost and an expected reduction in remaining uncertainty, and the selected path balances the two.

The interleaving of internal reasoning, environment observations, and state-changing operations is shown in Figure 14. ReAct represents action as a sequence in which reasoning traces guide external operations and observations update the context for the next decision [229].

3.1.6. Cross-Task Self-Improvement

Cross-Task Self-Improvement definition. Cross-Task Self-Improvement is a persistent change in a system’s later-task decision process that is produced from experience collected in completed trajectories. Persistence means that the retained information, learned procedure, routing behavior, verifier behavior, or model state survives the completed trajectory and remains available when a subsequent task begins. The later decision can concern reference retrieval, prompt construction, tool selection, action ordering, verification, repair, or another task operation. Action covers corrections performed during the active trajectory. Memory covers information retained for later access. Cross-Task Self-Improvement covers the update that makes retained experience alter a later task decision. Within-action correction operates inside one selected operation. Within-trajectory iteration uses observations and feedback before the current request ends. Episodic cross-task reuse retrieves a prior case or trajectory for a subsequent request. Persistent system adaptation changes a reusable skill, routing rule, construction policy, verifier, or model behavior across subsequent requests. Final artifact quality is an outcome measure. Candidate selection is a decision inside the active trajectory. A later-task behavior change is the defining unit of cross-task self-improvement. The boundary separates two improvement branches: foundation-model improvement updates the model itself through parameter changes, and scaffolding improvement updates the surrounding components such as prompts, memory, tools, and executable agent logic, so the locus of change determines whether the update carries the cost of retraining or the cost of retrieval. The recent survey of self-evolving AI agents organizes this landscape along three dimensions, model-centric evolution (inference-based and training-based), environment-centric evolution (static knowledge, dynamic experience, modular architecture, and agentic topology), and model-environment co-evolution, which align with the two branches above and place visual-generation self-improvement within a broader taxonomy [230].

Figure 15 presents a cross-task improvement loop in which exploration generates skills, execution feedback refines them, and a skill library supports later tasks. Voyager combines an automatic curriculum, iterative prompting, self-verification, and executable skill reuse across Minecraft tasks [231].

Experience formation and attribution. Experience formation and attribution covers the conversion of a completed trajectory into reusable information and the linking of observed outcomes to the actions that preceded them.

Experience formation converts a completed trajectory into information that can guide later decisions. The process records the task specification, references, intermediate artifacts, editable representations, tool calls, execution status, observations, diagnoses, verification results, critiques, repairs, costs, and user or evaluator judgments, together with provenance and task context. The formation process selects records with future decision value and compresses them into a case, a failure explanation, a repair relation, a preference, or another usable learning signal. Attribution links the observed outcome to the actions, conditions, and evidence that preceded it. This link attributes the observed improvement or failure to earlier choices, tool paths, repairs, and their conditions, and identifies the part of that relation that can be reused on a later task. The mechanism can compare successful and unsuccessful trajectories, align intermediate evidence with later outcomes, and discard records whose quality or provenance is unresolved. Attribution instantiates the credit assignment problem: it determines how the success or failure of an overall trajectory is distributed among the many earlier decisions that produced it, and because agentic trajectories span tens to hundreds of turns, episode-level attribution becomes uninformative and step-level signal must be recovered from outcome-level feedback. Agent-R addresses this recovery by using Monte Carlo tree search to splice a failed trajectory at its first error step with an adjacent correct path that shares the same parent node, so the model learns to reflect at the pivotal point where the deviation occurred rather than from the episode-level reward alone [232].

GenEvolve compares multiple tool-orchestrated image-generation trajectories for the same request and extracts best–worst differences into structured visual experience covering search, reference selection, prompt–reference construction, and failure avoidance; the experience is supplied to a teacher branch that distills guidance into a student policy [43]. SIDiffAgent records complete diffusion-agent trajectories, summarizes pitfalls and successes at decision nodes, retrieves similar records, and injects corrective and workflow guidance into later prompts; its second episode uses the accumulated database on later benchmark prompts [233]. These systems show experience formation as a sequence of logging, outcome attribution, abstraction, and later guidance construction.

Reusable representations. Reusable representations covers the forms in which retained experience is stored and the operational role each form makes available to a later task.

Reusable experience has several representation forms with different operational roles. A case preserves an episode, an artifact, a repair, or a tool trace for retrieval or replay on a related request. A strategy compresses recurring relations into a rule about search, reference choice, prompt construction, verification, or repair order. A skill packages a callable procedure, code fragment, workflow, or interface with inputs and execution conditions. The representation controls the amount of context carried into a later task and the degree of direct execution available to the system. Case reuse retains task-specific detail. Strategy reuse retains an abstraction that can guide several related tasks. Skill reuse exposes an executable operation that can be invoked as part of a later workflow. These forms instantiate the retention stage of the retrieve-reuse-revise-retain cycle: a solved problem is stored as a case or a compressed rule, the case enters a case base, and the case base makes the retained solution retrievable for a later similar problem, so the cycle closes only when retention has made the experience addressable for retrieval.

A long-video composition procedure can specify how clips, references, timing, and continuity checks are assembled. An image-editing correction rule can map a recurring defect to an edit operation and its parameters. A data-construction feedback rule can convert repeated rejection causes into the retrieval and synthesis choices for the next construction round. VideoWeaver refines and merges composition and creator skills from execution-trace and final-video evaluation, then tests the evolved skills on held-out task categories [70]. COMFYCLAW distills trajectories, execution errors, and verifier feedback into a progressively disclosed skill library for ComfyUI workflow construction and validates the library on held-out prompts [45]. The two systems instantiate skill-level reuse with different visual products and workflow interfaces.

Retention, retrieval, and adaptation locus. Retention, retrieval, and adaptation locus covers what survives a trajectory, how a later task queries it, and what changes after retrieval or learning.

Retention representation, access mechanism, and adaptation locus describe different parts of the update process. Retention representation specifies what survives a trajectory, such as an episode, summary, rule, skill, workflow, feedback record, or parameter state. Retrieval specifies how a later task queries that information, using signals such as task similarity, degradation-aware state fingerprints, artifact properties, tool compatibility, or temporal scope. Adaptation locus specifies what changes after retrieval or learning: the prompt, tool route, plan, verification rule, construction policy, skill library, or model parameters. An episode can therefore be stored in memory, retrieved by a state fingerprint, and applied as a prompt-construction change. A skill can be stored in a library, invoked through an interface, and used to alter tool ordering. A routing record can be retrieved from prior tool outcomes and used to select a generator. The retention form determines the adaptation channel: a record retained as context enters the later prompt without changing the model, a record retained as a skill enters the interface set as a callable capability, and a record retained as parameters enters the model weights through an update that persists across tasks, so the locus of retention fixes whether the later decision is changed by retrieval, by invocation, or by weight update. SEAL provides the parameter-level endpoint of this spectrum: the model generates a self-edit, a generation that may restructure information, specify optimization hyperparameters, or invoke tools for data augmentation, and the self-edit is applied through supervised finetuning as a persistent weight update; the self-edit policy itself is trained by reinforcement learning with the updated model’s downstream performance as the reward, so the adaptation loop is closed entirely within the model’s own generation and update [234]. AgentGym supplies the cross-environment analogue: a trinity of diverse environments, a trajectory set for prior capability, and an AgentEvol method that evolves the agent beyond previously seen tasks and environments, which places the parameter-level locus under explicit transfer and generalization tests [235].

OctoT2I updates a text-to-image router through repeated evaluation and learning, retaining tool-use knowledge that changes later routing and supports new-tool integration [44]. EvoIR-Agent organizes restoration experience in a hierarchical pool and retrieves it at multiple levels to guide later diagnosis and tool ordering [236]. SEAR indexes explored restoration trajectories with degradation-aware state fingerprints and reuses the resulting episodic memory to reduce later search cost [220]. DataEvolver retains verifier scores and rejection causes, summarizes them as semantic feedback, and feeds the updated feedback memory into later retrieval and targeted synthesis; the resulting construction data transfers to a second evaluated generator [216]. These cases place the update at different loci while preserving the same sequence of retention, access, and later decision change.

Transfer and evaluation. Transfer and evaluation covers whether a retained experience changes a later task and whether the change remains useful under new conditions.

Evaluation determines whether a retained experience changes a later task and whether the change remains useful under new conditions. In-task improvement measures the quality change across revisions of one request. Cross-task transfer measures later requests after experience construction. Generalization tests unseen prompts, task families, tool combinations, generators, executors, or related environments. Persistence tests whether the update remains available after the original trajectory and across later sessions or batches. Safety evaluation measures forgetting, harmful updates, evaluator overfitting, retrieval contamination, and regressions on previously verified capabilities.

A causal comparison holds fixed the factors outside the claimed adaptation mechanism and varies the retained experience or update. The trace records whether the later run retrieves a different reference, selects a different tool or order, constructs a different prompt, changes a verification threshold, chooses another repair, or takes another related decision. Artifact evaluation measures constraint satisfaction, semantic alignment, structural validity, visual quality, and domain-specific properties. Process evaluation records calls, latency, cost, retries, recovery steps, and stopping decisions. Held-out task families, unseen tool combinations, independent evaluators, and human judgments test transfer beyond the trajectories used to build the update. Ablations with matched generators and sampling budgets separate retrieval effects, extra sampling, generator strength, and persistent policy or skill changes. Four failure modes bound the evaluation. Parameter-level updates face catastrophic forgetting: a weight update that optimizes the new task pushes parameters away from the values that were optimal for earlier tasks, so the stability-plasticity dilemma requires a balance between retaining old knowledge and learning new concepts. Experience reuse is not monotonically beneficial: when prior experience conflicts with the new task’s conditions, negative transfer reduces rather than improves later performance. Verifier-driven improvement faces reward hacking: the agent overoptimizes the verifier’s proxy reward without satisfying the latent objective, so the proxy score rises while genuine quality stalls. When the evaluation set overlaps the experience-construction set, retrieval contamination inflates the score and masks whether the update generalizes or merely retrieves a stored answer. Self-Rewarding Language Models instantiate the verifier-driven loop and its risk: the model uses LLM-as-a-Judge prompting to assign its own rewards during iterative DPO training, and the reported result is that both instruction following and the reward-modeling ability improve together across iterations, which is the intended regime; the same design also exposes the reward-hacking surface, because the verifier and the policy are the same model and a degenerate self-edit can raise the proxy reward without raising genuine quality [237].

VISTA supplies a test-time boundary case in video generation. Its procedure builds a temporal plan, generates candidate videos, selects a candidate through pairwise comparison, collects visual, audio, and contextual critiques, and rewrites the prompt for later rounds; its evaluation compares successive rounds on single- and multi-scene tasks [64]. The reported adaptation occurs inside the test-time process. Persistent cross-task evidence uses a later-task retention test, transfer split, forgetting check, provenance record, and reversible update procedure.

4. Agentic Image Generation, Editing, and Restoration

Image creation agents must control an evolving artifact whose relevant state can include composition, masks, object identity, references, edit history, degradation estimates, user preferences, and tool outputs. Section 3.1 identifies the analytical components used to compare these systems. This section instead organizes image-domain studies by the primary contribution that they demonstrate. Each paper is assigned one location; secondary mechanisms remain relevant for cross-cutting comparison but do not determine a second placement.

Figure 16. Representative systems and benchmarks for agentic image generation, editing, and restoration, organized according to the chapter taxonomy.

Figure 16. Representative systems and benchmarks for agentic image generation, editing, and restoration, organized according to the chapter taxonomy.

4.1. Feedback-Driven Image Generation and Editing

Feedback-driven systems observe a generated artifact, identify a deviation from the current goal, and revise a later generation or editing decision. The following subsections distinguish compositional and design refinement from image-level editing, completion, and restoration.

4.1.1. Compositional Generation and Design Refinement

These systems use generated images as evidence for revising compositional constraints, design intent, or local defects. Their common contribution is a feedback action that changes the current generation trajectory.

Improving Compositional Text-to-Image Generation with Large Vision-Language Models.

This work uses an LVLM as both evaluator and editor for compositional text-to-image generation. The LVLM converts prompts into QA checks, uses answer accuracy as a loss weight for ReFL fine-tuning, and at inference iteratively detects mismatches and directs SAM+Blended Diffusion to correct them until fully compliant. [253]

Idea2Img.

is a GPT-4V agent that iteratively refines prompts for multimodal image design without prior knowledge of the T2I model’s optimal usage. Each round, GPT-4V generates candidate prompts, selects the best draft, reflects on mismatches with the user IDEA (interleaved text and images), and stores feedback in memory to guide revision. On 104 queries, iterative SDXL v1.0 wins 56.7% (vs. 29.8% initial, 13.5% manual; +26.9 SDXL, +16.3 IF). Memory guides revision within runs; cross-run retention is untested. [15] Figure 17 illustrates how retained comparisons guide successive prompt revisions.

MuLan.

is a training-free agent that decomposes complex prompts into object-level subtasks, generating each object conditioned on prior ones via LLM-planned masks and attention guidance, with a VLM verifying each subtask and triggering regeneration on violation; users may intervene mid-process. VLM feedback proves critical. [16]

VisualPrompter.

is a training-free prompt optimization framework that fixes semantic omissions in text-to-image generation by decomposing prompts into atomic concepts, using a VLM to identify which concepts are missing in the generated image, and revising only those units before reassembling the prompt. [254]

Maestro.

is a test-time agent system that automates T2I prompt refinement from underspecified user prompts. It decomposes prompts into DVQs, applies MLLM-driven targeted revisions with semantic verification, and maintains the best image via pairwise MLLM tournaments. [255]

CRAFT

is a training-free framework that improves T2I generation via constraint verification. It converts prompts into visual questions, uses a VLM to identify failures with rationales, and applies targeted LLM edits only where needed, retaining the best image and stopping once all constraints pass. [17]

Iterative Refinement Improves Compositional Image Generation.

This work proposes iterative refinement for compositional T2I generation, where a VLM critic inspects the current image and selects among continue, restart, backtrack, or stop under a fixed budget, optionally combined with parallel sampling [256].

Agentic Flow Steering.

AFS-Search is a training-free closed-loop framework on FLUX.1-dev for spatially grounded T2I generation. It rewrites prompts into explicit constraints, diagnoses intermediate latents, and compares three rollouts (baseline, exploration, and corrective via CLIP+SAM3 velocity modulation), selecting the best by VLM score. [257]

FiRe.

is a fine-grained reasoning framework for T2I generation that decomposes prompts into atomic tuples, verifies each via VQA, and applies localized corrections only where needed; FiRe-GRPO supplies step-level rewards. [18]

Self-Reasoning Agentic Framework for Narrative Product Grid-Collage Generation.

This framework generates product-advertising collages by decoupling narrative reasoning from pixel synthesis: agents plan a Product Narrative Framework (identity/usage/context/consumer), specify photographic decisions, and jointly synthesize the grid; a two-gate critique triggers targeted replanning or refinement on failure. [258]

4.1.2. Iterative Image Editing, Completion, and Restoration

This group places the feedback loop directly on an existing image. The controller diagnoses a mismatch, completion gap, or localized defect and selects a later editing or restoration operation.

An LLM–LVLM Driven Agent for Iterative and Fine-Grained Image Editing.

RefineEdit-Agent is a training-free agent for multi-step editing: an LVLM parses instructions into subgoals, an LLM selects/parameterizes tools (ControlNet, GLIGEN, InstructPix2Pix, etc.), and an LVLM evaluates fidelity/preservation after each edit, feeding back for re-planning until a threshold or budget is met. [238]

MIRA.

is a lightweight VLM agent that decomposes complex editing instructions into atomic steps via an iterative perception-reasoning-action loop: it observes the current image and instruction, emits one atomic edit or STOP, executes it via an external editor, and repeats until termination. [19]

JarvisEvo.

is a self-evolving photo-editing agent that jointly trains editor and evaluator policies via synergistic policy optimization on 170K Lightroom-style editing traces. It uses interleaved multimodal reasoning and self-generated reflection data to refine operations across 200+ tools. [259]

Agentic Retoucher.

is a perception–reasoning–action agent that detects and corrects localized defects (hands, faces, text, geometry) in T2I outputs. A perception agent predicts distortion saliency masks, a reasoning agent diagnoses defect types, and an action agent selects mask- or instruction-guided inpainting; the loop repeats until artifacts are resolved, typically within 2–3 iterations. [239]

EditRefiner.

is a four-agent perception–reasoning–action–evaluation loop that detects and corrects local artifacts in text-guided image edits. A perception agent predicts artifact/failure saliency maps, a reasoning agent diagnoses flaw types, an action agent performs masked re-editing, and an evaluation agent scores quality and decides whether to continue. The loop converges in 2.5 iterations. [240]

4.2. Planning and Tool Use for Image Creation

Planning and tool-use systems make an image task executable through model routing, external knowledge, structured controls, toolpaths, and task reformulation. The distinction below is between choosing a grounded plan and constructing an operational workflow.

4.2.1. Grounded Planning, Model Routing, and Structured Control

Planning-oriented systems first make missing knowledge, spatial constraints, model capabilities, or control variables explicit. Some also revisit the artifact after rendering, whereas others remain pre-generation routing systems.

DiffusionAgent.

routes text-to-image requests to domain/style specialist models. An LLM parses the input (instruction, inspiration, or hypothesis), searches a subject–style Tree-of-Thought model index, consults an advantage database built from 10,000 prompts and human feedback, selects an expert model, and extends the prompt from model-specific examples. [20]

Divide and Conquer.

CompAgent is an LLM-driven training-free framework for compositional T2I (text-to-image) generation. It decomposes prompts into objects/attributes with bounding boxes, selects among customization, layout-to-image, and localized editing tools, and uses multimodal inspection to repair incorrect attributes/relations. [8]

IA-T2I

retrieves reference images from the web when T2I prompts contain uncertain, rare, or ambiguous knowledge. An active retrieval module decides whether to search, a hierarchical selector ranks results, and self-reflection scores outputs and triggers re-retrieval/generation if needed. [260]

World-To-Image.

grounds T2I generation with web-retrieved knowledge for prompts involving novel/long-tail entities. An orchestrator diagnoses concept risk and selects among semantic decomposition, substitution, or image retrieval to condition OmniGen2. [261]

GenAgent.

is an agentic multimodal model that iteratively reasons, generates, judges, and reflects over multiple-turn interactions, treating image generators as callable tools until the output satisfies the request. [21]

Mind-Brush.

is an agentic framework that turns T2I generation into a dynamic and knowledge-driven workflow, Simulating a human-like ’think-research-create’ paradigm: it detects knowledge/reasoning gaps from user intent, routes them to web/image search or CoT reasoning, accumulates evidence, and compiles a master prompt for final generation. [262]

DiffGraph.

is an agent-driven framework that merges online expert models (checkpoints and LoRAs) for diverse T2I prompts. A graph construction agent registers expert descriptions as nodes and calibrates their capabilities on reference prompts as edge features; at inference, an expert selection agent retrieves relevant experts and a VGAE predicts merging coefficients from the activated subgraph, producing a merged model for rendering. [263]

Gen-Searcher.

is the first trained search-augmented image generation agent. It performs multi-hop web search, image search, and browsing over multiple turns, then emits a grounded prompt with ordered reference images for a single generator call. [22]

MetaPoint.

enables precise spatial control in UMMs via special tokens that reuse native 2D coordinate encodings: one token specifies a point, two define a box, and sequences encode layouts or edit regions; a VLM planner localizes objects and issues executable commands. [211]

One Image is All You Need.

WMGen-v1 generates long-tail training data for spatial perception from a single reference image: a Large Vision-Language Model (LVLM) extracts scene semantics, camera properties, and spatial constraints; an LLM expands these under physical/commonsense guidance; a diffusion model renders synthetic variants. [264]

RS-Gen.

is a training-free multi-stage agent for knowledge-intensive T2I generation/editing. An image router resolves coreferences against dialogue history; an intent analyzer routes to direct generation or structured questioning; a ReAct-based reasoning-and-search agent invokes web/image/geographic/VQA/reasoning tools with fallback retrieval; a generation agent executes a generate–verify–correct loop. [265]

Qwen-Image-Agent.

bridges the context gap in T2I generation by constructing full generation context from underspecified user input via three-tier planning (information/content/generation-level) and four-source grounding (reasoning, search, memory, feedback). Its unified context-centric design enables multi-image and multi-turn generation while preventing context drift. [215]

4.2.2. Workflow Construction and Task Reformulation

These systems choose and sequence heterogeneous image operations. Their primary object of control is a toolpath, workflow graph, or reformulated editing task rather than a single renderer call.

CAISE

is a dataset and task for conversational image search and editing, where an agent generates executable commands (search, color/brightness/contrast/rotation/background removal) from dialogue and visual context. The dataset contains 1,611 dialogues and 6,173 commands; the baseline generator-extractor model achieves 46.43% exact accuracy vs. 90.0% for human experts. [266]

GenArtist.

is a unified image generation/editing system driven by an MLLM agent. It decomposes complex prompts, builds a planning tree of generation/editing/auxiliary nodes with visual verification after each generation, selects tools from a heterogeneous library, supplies missing positions via detectors/segmentation/pose-depth preprocessors, and backtracks to sibling tools on failure. [23] Figure 18 shows how the planning tree connects verified subtasks to heterogeneous generation and editing tools.

RestoreAgent.

is an MLLM-driven agent that selects task order and models for mixed image degradations (denoising, deraining, dehazing, deblurring, JPEG removal, low-light enhancement). A vision encoder + Llama3 controller predicts the next tool from current image and history, with step-wise reassessment and rollback on failure. [267]

AgenticIR.

is an LLM/VLM-driven system that restores images with multiple interacting degradations (noise, blur, compression...) by composing specialist models via a perception–scheduling–execution–reflection–rescheduling loop; failed operations trigger rollback and re-planning. A fine-tuned DepictQA provides degradation descriptions and success judgments; offline exploration enumerates tool sequences on held-out images and summarizes success patterns into retrievable documents for scheduling. [24] Figure 19 illustrates this artifact-conditioned reflection, rollback, and rescheduling loop.

HybridAgent.

is an agentic image restoration system with a FastAgent for direct prompts (routing tools in 12% of SlowAgent time) and a fine-tuned SlowAgent for ambiguous requests; a FeedbackAgent judges cleanness and can trigger further restoration. Tools are built via three-stage training (base model → LoRA single-task tools → mixed-degradation tool), enabling joint restoration to replace damaging long chains. [268]

CoSTA*.

is a cost-sensitive agent that plans and executes multi-turn image edits via LLM-driven subtask tree generation, tool subgraph construction over 24 tools, and A* search balancing quality and runtime with a tunable cost-quality parameter; a VLM validator checks each subtask and triggers retries or alternative paths on failure. [10]

ComfyGPT

is a multi-agent system that generates executable ComfyUI workflows from natural-language task descriptions. ReformatAgent converts JSON workflows to link-centered diagrams, FlowAgent predicts diagrams (trained with SFT + GRPO with validity rewards), RefineAgent retrieves current node definitions from a database to replace invalid nodes, and ExecuteAgent submits the final workflow. [269]

Q-Agent.

is a quality-driven image restoration (IR) agent that addresses multiple degradations via CoT-based perception and greedy IQA-guided restoration. A fine-tuned MLLM answers per-degradation Yes/No questions; at each step, all candidate operations are applied, ranked by an aggregate of five NR-IQA metrics, and the best is greedily selected until no improvement. [270]

FaSTA*.

extends CoSTA* by mining reusable tool sequences (subroutines) from prior traces via LLM-based inductive reasoning and storing them with activation conditions in a rule table. It first attempts fast planning using these rules, with VLM checks validating each subroutine; only when no rule applies or a check fails does it fall back to localized A* search. [228]

Restore-R1.

is a lightweight reinforcement-learned agent that predicts restoration tool sequences in a single forward pass, eliminating runtime reflection/rollback. A CLIP+MLP policy is trained via PPO/GRPO using DeQA-Score as the reward, without ground-truth labels. [271]

Derain-Agent.

is a plug-and-play post-deraining enhancer that refines initial derained outputs by predicting one of 16 predefined tool paths (denoising, deblurring, color correction) and pixel-wise strength maps from a ResNet34 feature extractor, then executing the path once. [272]

TIR-Agent.

is a trainable VLM agent that learns direct tool-calling policies for multi-degradation restoration via SFT + RL, replacing costly heuristic search. A Qwen3-VL-8B controller selects task and model per step from current image and history; EDP diversifies SFT trajectories, while MAR dynamically reweights FR/NR metrics to prevent reward hacking. [241]

IMAGAgent.

is a plan–execute–reflect agent for multi-turn image editing that prevents error accumulation and semantic drift. A VLM planner decomposes instructions into atomic sub-tasks; a controller dynamically composes segmentation, detection, retrieval, and editing tools from the current image and history; multiple VLM experts critique intermediate outputs; and an aggregator converts feedback into corrective actions, with up to three retries per turn. [214]

Adaptive Task Reformulation.

ATR improves image editing by reformulating ambiguous tasks without modifying the backbone. A profiler extracts target and scene context; a router selects direct/rewritten editing, spatial decoupling, or localized workspace; a planner executes sequentially with state, feedback, and fallback. [213]

From Plans to Pixels.

is an experiential learning framework for long-horizon advertisement editing. A checklist-guided planner decomposes abstract instructions into subtasks; a reward-trained orchestrator selects tools/regions from a heterogeneous editor library; a VLM judge provides outcome rewards, infeasible subtasks are pruned, and a distilled verifier re-ranks intermediate candidates. [212]

OPERA

is an end-to-end framework for multi-tool cooperative image restoration. A GRPO-trained planning agent emits complete tool compositions directly from the input, receiving final restoration quality as reward; 16 restoration tools are jointly fine-tuned through generated chains, enabling cooperative behavior. [273]

GenClaw.

is a code-driven agentic image generation framework that decouples creation into Conceptualize–Sketch–Color stages: it retrieves contextual knowledge via search/reasoning, constructs executable visual code (SVG/HTML/Canvas/Three.js) as an intermediate layout, then renders photorealism via an image generator. [221]

DiTTo.

is an order-aware restoration agent framework that decouples training from real-expert calls via a simulator (US-IR + AiO-IQA) that reduces ORTD construction from $\mathcal{O}\left({\left(N^{\mathrm{D}}\right)}^2\right)$ to $\mathcal{O}\left(N^{\mathrm{D}}\right)$ calls. A VLM agent is SFT-trained on simulated trajectories, then aligned to a small real-expert set via decomposed DPO (DP/OR/Tool axes); new experts require updating only the alignment stage. [274]

IEA.

is an amateur-friendly conversational editor that performs global photo retouching via 16 parameterized tools (brightness, exposure, contrast, etc.), emitting explicit tool calls rather than synthesizing pixels. A Qwen2.5-VL-7B policy is trained in three stages: Stage 1 distills expert programs from GIER for SFT; Stage 2 applies GRPO with likeness and usefulness rewards; Stage 3 adds synthetic Image-Edit, Image-Summary, and Image-Refine data to enable history summarization and feedback-driven refinement. [275]

CanvasAgent.

is a tool-augmented agent for complex multi-step image workflows across 11 tools (generation, edit, grounding, etc.). Trained via SFT + GRPO on CanvasCraft (140K SFT trajectories + 10K RL tasks), it maintains asset state, supports plan revision and rollback. SFT+RL substantially outperforms SFT-only across overall reward, alignment, and trajectory quality; human evaluation also favors it over Qwen3-VL-32B. [25]

4.3. Stateful Image Generation and Editing

Stateful image systems retain information needed to preserve identity, user intent, scene structure, and earlier decisions. The literature separates turn-level user and context management from explicit scene, narrative, and trajectory representations.

4.3.1. Multi-Turn User, Subject, and Context Management

These systems retain information that is needed across turns, including user preferences, subject identity, prior operations, context, and candidate quality. The persistent representation makes a later action dependent on more than the current prompt.

AutoStudio.

is a training-free multi-agent framework for multi-turn interactive image generation. It stores subject features with persistent IDs; a subject manager parses dialogue, a layout generator predicts bounding boxes, a supervisor refines spatial relations, and a P-UNet drawer renders the final image via subject-initialized generation. [26]

Preference Adaptive and Sequential Text-to-Image Generation (PASTA)

is an RL agent for multi-turn T2I generation that iteratively refines prompts via adaptive expansion slates guided by user selections and a learned preference model. Trained on 7,000 human rater sequences + 30,000 simulated rollouts, its user model achieves 70% accuracy on held-out metrics. [27]

StoryState.

is an agent-based state controller for consistent and editable storybooks. It externalizes each story as $S=(C,W,\left\{S_i\right\})$—a character sheet, global world state, and page-level scene states—maintained by Planner and State Manager agents. A Prompt Writer maps this state to global/page prompts; local edits update a single $S_i$, identity edits update C and propagate to affected pages; a Consistency Critic verifies outputs against the state and neighboring pages. [28]

Agent Banana.

is a hierarchical planner-executor agentic framework for professional image editing that addresses over-editing, multi-turn fidelity loss, and native 4K resolution. It decomposes vague instructions into atomic operations, performs localized editing via Image Layer Decomposition to preserve non-target regions, and uses Context Folding for stable long-horizon state tracking, with self-reflection enabling retry, rollback, and replanning. [219].

Generation Navigator.

frames multi-turn text-to-image generation as a state-conditioned action-making problem, where a learned multimodal navigator dynamically chooses to STOP, REFINE, or REGENERATE based on the current image and reviewer feedback [29]. To train this policy, the system constructs 103K trajectories and applies PRE-GRPO, a trajectory-level reinforcement learning objective that jointly rewards peak quality, retention against degradation, and turn efficiency.

4.3.2. Scene, Narrative, and Trajectory State Management

For multi-object edits and visual narratives, systems externalize scene structure, causal relations, candidate branches, or operation histories. This turns an image sequence into an addressable state space for planning and recovery.

I2E

reformulates compositional image editing as interaction within a structured environment, using a Decomposer to segment objects, recover occlusions, and build physically ordered layers via DAG-based spatial propagation [242]. A physics-aware VLA Editor translates instructions into atomic actions (REMOVE, MOVE, FALL, RESIZE, EDIT, INSERT) through chain-of-thought reasoning, executing object-level edits without global pixel resampling.

MSRAMIE

is a training-free multimodal agent framework that handles complex multi-instruction image editing by decomposing lengthy requests into structured sub-tasks through iterative interactions between an MLLM-based Instructor and a plug-in editing Actor [243]. It constructs a Tree-of-States for flexible state transitions, backtracking, and resampling, while a Graph-of-References retrieves nearby states to avoid redundant exploration.

LogiStory.

targets multi-image story visualization by explicitly modeling visual logic—causal and perceptual coherence across characters, actions, and scenes over time [30]. A multi-agent system extracts entities, causal events, and panel scripts; a Local Causal Monitor checks each frame against accumulated narrative memory, while a Global Causal Verifier maintains a story-level causal graph and triggers regeneration or editing upon inconsistency.

4.4. Multi-Agent Systems for Image Generation and Editing

Multi-agent systems allocate visual creation work among specialized roles. Their value depends on how role outputs, shared state, and evaluator feedback determine later actions, rather than on the number of participating agents.

4.4.1. Collaborative Generation, Composition, and Domain Design

This group distributes generation, analysis, design, or evaluation work across specialized roles. It includes contextual comparisons that clarify the difference between model-level cooperation and task-level visual creation control.

Message Passing Multi-Agent GANs.

This early work explores unsupervised image generation with two generators that share a discriminator and exchange learned messages via a common message generator, with competing or conceding objectives encouraging complementary behavior [276].

Anywhere.

replaces end-to-end inpainting with a multi-agent pipeline for foreground-conditioned image generation, preserving object integrity and supporting optional text guidance [227]. A Foreground Analyzer and Prompt Creator generate scene prompts; a Template Repainter repairs mask violations; and a Quality Evaluator triggers regeneration when needed.

Marmot.

addresses counting, attribute-binding, and spatial errors in text-to-image generation via object-level divide-and-conquer [31]. An Object-Aware Agent decomposes the scene into subtasks; an Object Correction System with decision-execution-verification operates on individual masks or object-pair boxes, retrying upon failure; a Pixel-Domain Stitching Smoother merges corrected regions via mask-guided optimization.

MCCD

enhances complex text-to-image generation through a multi-agent parsing module and hierarchical compositional diffusion [277]. A conductor schedules six specialist agents with forward reasoning and evaluator-triggered backward feedback; the diffusion module renders parsed elements via depth-aware Gaussian masks, regional enhancement, and boundary smoothing.

T2I-Copilot.

addresses ambiguous text-to-image prompts through a training-free multi-agent system that interprets user intent, selects appropriate models, and iteratively refines outputs [32]. An Input Interpreter extracts entities, attributes, and ambiguities; a Generation Engine selects and executes the optimal generation or editing route; and a Quality Evaluator scores aesthetic and alignment criteria, triggering regeneration with structured suggestions when scores fall below a threshold.

From Image Generation to Infrastructure Design.

This work applies multi-agent image editing to bicycle-lane visualization as a constrained street-scene editing problem [278]. A Locator Agent identifies roadway geometry; a Prompt Agent formalizes user intent; a Design Generation Agent produces candidates via highlight-first cascading; and an Evaluator Agent reranks candidates and verifies hard constraints, triggering regeneration when none pass.

GenPilot.

optimizes text-to-image prompts at test- time via a multi-agent system that iteratively refines inputs to improve semantic alignment [279]. An error-analysis stage decomposes prompts and combines VQA with caption comparison to localize mismatches; a test-time optimization stage generates candidate rewrites, scores them with an MLLM, clusters candidates, and uses memory to guide subsequent rounds.

Collaborative Text-to-Image Generation.

This work explores multimodal text-to-image generation via domain-specialized agents (architecture, portraiture, landscape) coupled with PPO training and multimodal fusion [280]. A text-enhancement subsystem and an image-generation subsystem each contain specialized agents; PPO optimizes them with a composite reward over similarity, quality, and diversity, while contrastive learning and bidirectional attention enforce cross-modal alignment.

M3.

is a training-free multi-agent framework that refines text-to-image outputs through iterative, validated editing [244]. A Planner decomposes prompts into checklists; a Checker evaluates each constraint; a Refiner generates edit instructions for failures; an Editor executes edits; and a Verifier accepts only edits that improve alignment over the previous best.

coDrawAgents.

is a multi-agent framework for compositional text-to-image generation where an Interpreter adaptively chooses between direct generation and layout-aware mode [245]. In layout-aware mode, a Planner incrementally proposes placements for priority-grouped objects grounded in the evolving canvas, a Checker validates and refines spatial and semantic consistency, and a Painter synthesizes the partial image to inform subsequent iterations.

InterleaveThinker.

enables long-horizon interleaved text-image generation by decoupling global planning from stepwise execution through a multi-agent framework [281]. A Planner produces an N-step plan upfront; at each step, a frozen image generator executes the instruction, and a Critic evaluates the result against the plan, issuing binary judgments and refined prompts. The Critic is trained with dual-reward GRPO combining judgment accuracy and correction quality.

AuDiffusion.

is a multi-agent diffusion framework with Prompt, Layout, and Executor agents for controllable text-to-image generation [282]. The Prompt Agent enriches semantics; the Layout Agent selects ControlNet modules via capability-requirement matching; the Executor synthesizes images using an AuMamba backbone with linear-time state-space scans and windowed attention. A shared blackboard stores states and triggers targeted revisions when quality falls below threshold.

Multi-agent Collaborative Pathways for Chinese Traditional Architectural Image Generation.

Problem and setting. This system generates images of Chinese traditional architecture from non-specialist requests while attempting to preserve historically specific form, color systems, decorative symbols, and cultural meaning [283]. Method and mechanism. A domain knowledge base encodes entities, architectural components, spatial rules, symbolism, and historical versions. Intent understanding and prompt-generation agents ground vague requests in this knowledge; a Stable Diffusion LoRA plus Canny ControlNet renders the image. An aesthetic and cultural-relevance agent returns scores and error flags to a workflow scheduler. Depending on the diagnosed cause, the scheduler revisits prompt generation, changes rendering parameters, or returns to intent interpretation, with a maximum iteration count preventing unbounded execution. Evaluation and results. Sixty participants, comprising 20 domain experts, 20 relevant students, and 20 general users, score anonymized outputs. The MAS prototype reports $4.65\pm 0.25$ for architectural form, $4.72\pm 0.20$ for color fidelity, and $4.71\pm 0.22$ for decorative-symbol consistency, compared with $4.50\pm 0.28$, $4.58\pm 0.26$, and $4.61\pm 0.29$ for the same generator driven by expert-authored prompts. Its average CLIP score is $0.89\pm 0.03$, compared with $0.86\pm 0.04$ for that baseline. Evidence boundary. Visual and cultural error reports determine targeted rerouting, providing L3 artifact-conditioned correction. The evidence comes from one cultural case study and an author-built knowledge base; CLIP against the optimized prompt is not a direct measure of historical truth, and the evaluation does not isolate the contribution of repeated correction from knowledge grounding and model specialization. A single pass averages about 166.7 seconds, with additional iteration cost.

MUSES

generates 2D images with precise 3D control by leveraging 3D layouts, models, and rendering as intermediate guidance [33]. A Layout Manager plans 2D layouts via in-context learning and lifts them to 3D (depth, orientation, camera) via chain-of-thought; a Model Engineer retrieves or generates 3D assets and aligns them to camera using a fine-tuned CLIP classifier; an Image Artist assembles the scene in Blender and renders depth/Canny controls for final diffusion. Figure 20 shows how typed 2D and 3D intermediates coordinate these specialized agents.

4.4.2. Collaborative Editing, Completion, and Restoration

These systems divide image modification into complementary planning, execution, evaluation, and specialized restoration or completion roles. Their evidence concerns whether this division improves an evolving edited artifact.

CCA

addresses complex image editing by deploying two generator agents that independently decompose instructions and execute tools, and a discriminator that compares outputs, provides subtask-level feedback, and triggers parameter adjustments or tool reselection across rounds [34]. A best-candidate bank enables cross-round comparison, with early stopping when quality meets requirements (up to five rounds).

MAIR

is a multi-agent system for complex image restoration, guided by a real-world degradation prior that reverses scene, imaging, and compression artifacts in sequence [246]. A Scheduler agent plans the overall restoration order using perception, textual experience, and user instructions; Expert agents then sequentially remove specific degradations by selecting from registered tools, executing, and reflecting on results before passing to the next.

CREA

is a multi-agent framework for creative image editing and generation [284]. A Creative Director sets the vision; a Prompt Architect fuses six creativity-principled prompts; a Generative Executor synthesizes images; a Critic scores six creative dimensions; and a Refinement Strategist adjusts prompts over up to three iterative rounds based on weak scores.

Talk2Image.

is a multi-agent system for multi-turn conversational image generation and editing, where later instructions accumulate with earlier scene content and affect only requested regions [247]. A multi-turn intention parser converts dialogue history into structured cumulative specifications; specialized agents execute generation, editing, segmentation, VQA, and chat via DAG-scheduled operations with blackboard-based coordination; and multi-view feedback scores semantic alignment and visual consistency, triggering retries until a threshold is met.

Multi-Agent Amodal Completion.

This multi-agent framework reconstructs occluded or truncated objects and outputs layered RGBA assets [285]. An Occlusion Agent identifies front-back relations, a Segmentation Agent extracts masks, a Boundary Agent estimates canvas expansion, and a Description Agent provides semantic guidance; these upfront decisions define a single inpainting mask and prompt for one-pass FLUX-ControlNet synthesis, with attention maps fused to generate the alpha channel.

ImageEdit-R1.

addresses complex, multi-step image editing instructions through a reinforcement learning-enhanced multi-agent framework [35]. A decomposition agent (Qwen2.5-VL-7B) parses user requests into actions, subjects, and goals; a sequencing agent orders sub-requests; a diffusion-based editor executes them. GRPO trains the decomposition agent with format, action, subject, and goal rewards without modifying the editor.

CAMEO

is a hierarchical multi-agent framework for conditional image editing that replaces single-pass generation with quality-aware, feedback-driven refinement [36]. A Strategic Director interprets requests and selects constraints; a Visual Research Specialist retrieves or synthesizes references; a Quality Critic evaluates intermediate outputs across task-adaptive dimensions and returns structured deviations; and a Refinement Editor applies targeted corrections, looping until thresholds are met.

APE

is a lightweight framework that post-trains small language models as prompt enhancers for image generation and editing without modifying downstream models [286]. Its single-agent (SAPE) and multi-agent (MAPE) variants decompose enhancement into router-selected field rewriting and composer fusion, trained via SFT and GRPO/GDPO on downstream rewards (PickScore, CLIPScore, HPSv2.1, GPT-based editing judgments).

4.5. Native and Unified Agentic Models for Image Creation

Native and unified models place reasoning, understanding, generation, and sometimes tool use within a common multimodal policy. This organization is discussed separately from the feedback and workflow mechanisms analyzed in earlier groups.

4.5.1. Native and Unified Models for Reasoning and Generation

Native and unified models integrate image understanding, reasoning, and generation in one learned policy. The grouping concerns the implementation locus of these functions, not an automatic claim of high autonomy.

ImAgent.

is a training-free unified multimodal agent that enables adaptive test-time scaling for image generation and editing through a policy controller that dynamically selects actions based on observation history [37]. Built on a single UMM, its action space includes direct generation/editing, CoT prompt enhancement, image-conditioned prompt revision, local detail refinement, Best-of-N sampling, and STOP; invalid actions fall back to direct generation.

UniReason 1.0.

unifies text-to-image generation and editing within a shared framework for requests requiring implicit world knowledge [248]. World-knowledge-enhanced textual reasoning expands underspecified prompts into explicit guidance before synthesis; the model then observes the generated image, reflects, and applies editing-style refinement to correct discrepancies. Training uses an agent pipeline (generator → verifier → refiner → judge) and two-stage SFT on reasoning and interleaved refinement samples.

VisionCreator.

is a native visual-generation agentic model unifying Understanding, Thinking, Planning, and Creation for complex image/video creation with multi-step tool invocation [38]. Progressive Specialization Training uses 4k expert-filtered trajectories, while Virtual Reinforcement Learning optimizes long-horizon planning in a simulated 36-tool environment with plan, format, execution, result, consistency, and trajectory rewards; average trajectories span 15 steps (64% >20).

VisionCreator-R1.

extends VisionCreator’s UTPC framework with Reflection, forming a UTPCR loop that inspects intermediate images, identifies deviations, and triggers targeted edits or regeneration [39]. Reflection-Plan Co-Optimization first learns reflection on single-image tasks, then mixes reflection-strong and planning-strong trajectories before multi-task RL with reflection, plan, format, tool, and result rewards.

4.5.2. Models for Grounded, Spatial, and Open-World Creation

This group extends unified policies to world knowledge, spatial reasoning, and tool-mediated open-world generation. It concentrates on models whose primary contribution is grounded creation within an integrated multimodal policy.

Unify-Agent.

is an end-to-end unified multimodal agent for world-grounded image synthesis, reformulating generation as a sequential process of prompt understanding, multimodal evidence search, grounded recaptioning, and final synthesis [40]. A unified model diagnoses missing knowledge, performs textual and visual search, and transforms retrieved evidence into a compact recaption trained on 143k annotated trajectories.

Boogu-Image-0.1.

is an open image-generation and editing model family designed for complex, multilingual, and text-rich requirements under constrained budgets [249]. Its core generator uses Qwen3-VL-8B as instruction encoder with a curated data syllabus; at inference, a DeepSeek-V4-Flash agent performs reasoning-oriented prompt rewriting and routes between Base and Turbo variants by task complexity, with reflection and Best-of-N as optional scaling.

ToolArtist.

is a fully agentic image-generation model obtained by post-training a unified multimodal model, where reasoning, external tool use, and native image generation are coordinated within a single policy [41]. Supervised trajectories are collected from a teacher agent with search and generation tools then converted to UMM-native format; RAD-GRPO jointly optimizes with intent and quality rewards.

4.6. Data, Evaluation, and Safety for Agentic Image Generation

Data construction, learning, evaluation, and safety research supports the development and assessment of image agents. These studies determine how controllers are trained, what claims of reusable experience are supported, and how quality or safety failures can be measured.

4.6.1. Data Construction, Policy Learning, and Reusable Experience

This group studies how visual-agent data, policies, workflows, and experience stores are constructed or improved. The central question is what, if anything, persists beyond a single trajectory and improves a later task.

Gen-n-Val.

is an agentic framework for generating synthetic instance segmentation data to address data scarcity and long-tailed imbalance [42]. A TextGrad-optimized LLM generates detailed prompts for Layer Diffusion to produce transparent single-instance foregrounds with alpha-channel masks; a VLM validator checks class identity, single view, object integrity, and plain background before accepted instances are composited into training scenes.

SIDiffAgent.

is a training-free self-improving diffusion agent that refines text-to-image generation via iterative prompt engineering, evaluation, and editing, supported by experience memory [233]. An orchestrator handles intent analysis and adaptive negative prompting; an evaluator triggers up to two corrections; a guidance agent stores and retrieves past trajectories to guide future generations.

Socratic-Geo.

is an autonomous multi-agent framework for geometry data synthesis and reasoning, where a Solver’s failed attempts trigger a Teacher to diagnose gaps, modify parameterized Python drawing code, and validate new problems via Reflect (solvability) and RePI (visual validity) before adding them to the curriculum [287]. A Generator is separately fine-tuned on accumulated image-code-instruction triplets to distill programmatic drawing into diffusion-based generation.

PaAgent.

is a portrait-aware image restoration agent that selects restoration experts via a self-evolving portrait bank with RAG, while subjective-objective RL with GRPO refines its degradation perception. [288]. A Qwen-based module perceives degradation using images, scores, and task history; RAG queries a bank of stored triplets to recommend the next expert, with each interaction added back to the bank.

ScaleEdit-12M

ScaleEditor is an open-source multi-agent framework for constructing large-scale image editing datasets without costly commercial APIs, producing 12M instruction-image-edit triplets across 23 tasks [289]. Source image expansion combines retrieval and synthesis; a task router assigns images to specialized editing agents; task-aware verification scores instruction following, editing consistency, and generation quality, retaining only high-scoring pairs.

GenEvolve.

is a self-evolving framework for open-ended image generation where an orchestrates factual search, visual reference retrieval, internal generation knowledge, and prompt-reference program synthesis along tool-based trajectories [43]. It uses SFT cold-start, GRPO for trajectory-level optimization, and Tool-Orchestrated Visual Experience Distillation: comparing best-worst trajectories, extracting structured experience slots, conditioning a privileged teacher on retrieved experience, and distilling token-level preferences into the student policy.

EvoIR-Agent.

is a self-evolving image restoration agent that reduces trial-and-error in selecting restoration order and tools by reusing experience [236]. A hierarchical experience pool stores guidance at three granularities: high-level insights, degradation-type mappings, and fine-grained pattern records; after batch accumulation, an evolution procedure updates entries and stabilizes priorities from quality rankings via Bradley-Terry-Davidson modeling, enabling later retrieval of outcome-reflected guidance.

OctoT2I.

is an agentic router that selects among T2I models to jointly optimize generation quality and inference efficiency, with a self-evolving knowledge base built without human supervision [44]. During inference, a stateful multi-round router consults long-term tool knowledge and task-local memory, selects a generator, evaluates alignment, and stores the score for subsequent decisions. A Propose–Solve–Evaluate–Learn loop defines conceptual dimensions, generates exploration prompts, measures each tool’s Pass@1, prunes mastered combinations, and updates empirical and semantic profiles.

A Task-Driven and Quality-Assured Agent Framework for SAR Data Generation.

SAGA is a schema-grounded agent framework for SAR data augmentation, conditioning generation decisions on dataset structure, task objectives, and verification evidence rather than fixed pipelines [290]. It profiles datasets into validated schemas, ranks plans via benefit-cost-risk scoring, compiles them into recipe DAGs, and evaluates outputs through observer checks with bounded repair; policy memory stores evidence for reuse.

Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Execution.

SEAR is a dual-process self-evolving agent for image restoration with coupled degradations, where a Deliberate Planner uses LLM scheduling and P-MCTS to explore long-horizon paths with hybrid rewards and MLLM tournament verification, while high-reward trajectories are distilled into episodic memory indexed by degradation fingerprints [220]. An Intuitive Executor retrieves stored strategies for recurring patterns, avoiding repeated search.

DataEvolver.

is a self-evolving multi-agent framework for text-rich image data construction, treating rejected samples as feedback for improving subsequent collection rounds rather than discarding them [216]. A Retriever gathers candidates; a Verifier filters by quality, OCR, semantics, and duplicates; a Critic summarizes rejection patterns into semantic feedback for updating retrieval queries and prompts; and a Generator fills under-covered regions.

COMFYCLAW.

is a self-evolving skill harness for image-generation workflows in ComfyUI, where agents must construct valid graphs, satisfy node/model constraints, and repair visual failures across recurring tasks without rediscovering fixes [45]. It exposes stage-gated tools for typed DAG editing and rollback; a region-level VLM verifier translates visual defects into repair suggestions; successful trajectories and errors are distilled into skills, validated on held-out tasks before library commitment.

Search Beyond What Can Be Taught.

This work uses external search to ground knowledge-intensive text-to-image generation, but finds that naive retrieval corrupts outputs by overriding known concepts or injecting noise [210]. It therefore co-trains a reasoner with a Gate-Filter-Integrate protocol to decide when and what to retrieve, alongside online DPO that expands the generator’s parametric knowledge and rejection-sampling fine-tuning that recalibrates the reasoner to search only for what remains outside.

4.6.2. Evaluation and Safety

Evaluation and safety studies provide the measurements and adversarial evidence needed to assess image-agent behavior. They are not treated as image-creation controllers merely because they use planning, search, or multiple roles.

A Unified Agentic Framework for Evaluating Conditional Image Generation.

This work proposes CIGEVAL, an agentic framework that evaluates conditional image generation by decomposing tasks into fine-grained sub-questions, autonomously selecting tools like Grounding, Difference, Highlight, and Scene Graph to inspect specific visual aspects, and producing explainable scores with rationales. [47].

EdiVal-Agent.

is an object-centric framework that evaluates multi-turn image editing by decomposing images into object pools, tracking their evolution across turns, and computing instruction-following (via detectors and VLMs on guided crops), content consistency (via DINOv3 similarity of unchanged objects and backgrounds), and visual quality (via HPSv3). [48].

Blueprint-Bench.

evaluates spatial reasoning by tasking models with converting apartment photographs into 2D floor plans, scoring outputs on room connectivity and size ranking [6]. Most LLMs, image generators, and agents perform at or below random baseline; human performance remains substantially superior, and iterative agent refinement does not meaningfully help.

Value-Aligned Prompt Moderation.

VALOR moderates unsafe text-to-image prompts via a zero-shot agent that layers lexical, semantic, and value-level detection with intention disambiguation, selectively rewriting harmful requests and optionally triggering style-guided regeneration when generated images remain unsafe [250].

OrchJail.

jailbreaks tool-calling text-to-image agents by exploiting orchestration-level vulnerabilities, where individually benign tool calls compose into policy-violating outcomes. It learns causal relationships between prompt phrasing and tool-execution patterns from successful jailbreak traces, guiding mutation and scoring to efficiently search for prompts that trigger unsafe multi-step behaviors. [251].

Whispers in the Noise.

This work reveals that concept erasure in diffusion models only disrupts early text-to-semantics mapping while leaving later noise-driven dynamics intact. ConceptAgent exploits this by injecting surrogate-preserved geometry and color into intermediate denoising states, bypassing the erased text-conditioning path. Experiments show it reliably awakens erased concepts across multiple erasure methods and backbones, outperforming baselines without training or model access [252].

RedEdit.

red-teams image safety classifiers by formulating photo-editing evasion as a combinatorial search over edit sequences. A VLM proposes candidate edits while MCTS plans promising paths and backtracks from ineffective ones, requiring both detector evasion and content preservation. Experiments reveals systemic vulnerabilities in current moderation systems [46].

Does AI Understand Imaging?.

ImagingBench evaluates multimodal agents on computational imaging tasks including reconstruction, sensing, optics, and calibration. Across 20 subtasks and three protocols, agentic systems consistently underperform specialized methods, especially on lensless imaging, event reconstruction, time-of-flight, and holography; planner guidance offers only modest gains, and visually plausible outputs often have poor reference-based fidelity [49].

These studies broaden assessment beyond final appearance to object-level edit correctness, spatial reasoning, physical fidelity, safety moderation, and verifier robustness. Their diagnostic outputs become evidence for AVG only when a separate controller uses them to choose a corrective action.

Across image tasks, corrective precision depends on whether the system retains an addressable representation of the artifact and connects verification evidence to a later action. Masks, object tables, edit histories, preference records, degradation estimates, and executable workflows can support localized revision; undifferentiated scores often lead to global resampling. The evidence remains uneven for verifier calibration, causal diagnosis, rollback, budget-aware stopping, and persistent improvement beyond one task.

5. Agentic Video, Film, and Animation Generation

Video generation extends both the planning horizon and the scope of each correction. Systems work with scripts, shots, keyframes, motion, audio, camera state, and timelines. Even a local edit may disrupt identity or causality several shots later. We therefore distinguish staged pipelines from systems that carry temporal state forward, locate failures, and revise the affected material. Figure 21 summarizes these mechanisms across the video-production lifecycle.

Figure 21. Overview of agentic video generation across the production lifecycle. Pre-production representations guide multi-shot generation and timeline editing; feedback and recovery revise failed local segments, while a persistent temporal state carries scripts, entities, shots, keyframes, motion, camera, audio, and timeline information across stages. Verified experience can further be abstracted into reusable skills for future tasks.

Figure 21. Overview of agentic video generation across the production lifecycle. Pre-production representations guide multi-shot generation and timeline editing; feedback and recovery revise failed local segments, while a persistent temporal state carries scripts, entities, shots, keyframes, motion, camera, audio, and timeline information across stages. Verified experience can further be abstracted into reusable skills for future tasks.

Figure 22. Representative systems and benchmarks for agentic video, film, and animation generation, organized according to the chapter taxonomy.

Figure 22. Representative systems and benchmarks for agentic video, film, and animation generation, organized according to the chapter taxonomy.

5.1. Scripts, Storyboards, Motion, and Pre-Production

Pre-production turns long-range intent into scripts, shot plans, storyboards, trajectories, or motion programs. These representations clarify what should happen, but downstream generators must still preserve the specified characters, settings, and constraints.

Anim-Director

uses an LMM as an autonomous director to turn a short narrative into a multi-scene animation. It expands the story into a structured script with character, setting, and scene descriptions, generates reference and scene images, and checks their consistency before using the selected images to guide video generation. The LMM also evaluates and selects image and video candidates during generation [50].

Agentic Aerial Cinematography

translates a director’s free-form language instruction into an executable indoor UAV video tour. It uses vision–language retrieval to select initial waypoints, refines camera poses through preference-based Bayesian optimization with aesthetic feedback, and plans safe, dynamically feasible quadrotor trajectories [291].

AnimAgents

supports animation pre-production through dedicated boards for ideation, design, scripting, and storyboarding. A stage-aware Core Agent coordinates specialized agents and carries relevant project context across stages, while creators can select and refine individual elements without regenerating entire outputs [51].

Automated Movie Generation

MovieAgent uses hierarchical multi-agent planning to turn a script synopsis and character bank into a structured multi-scene, multi-shot movie plan. A Director Agent organizes the narrative, while Scene Plan and Shot Plan agents progressively specify scenes, character interactions, camera movements, shot types, and dialogue before customized video and audio models generate the final shots [292].

Lighting-grounded Video Generation

LiVER enables controllable video generation by representing scene layout, lighting, and camera trajectory as explicit 3D conditions. A scene agent translates high-level user instructions into an editable 3D scene representation, whose rendered control signals are then used to condition a video diffusion model [293].

The Script is All You Need

ScripterAgent translates coarse dialogue into a fine-grained, executable cinematic script that provides shot-level guidance for subsequent video generation. DirectorAgent then executes the script through cross-scene continuous generation with frame anchoring, carrying visual context between scenes to improve long-horizon continuity [294].

Mind-of-Director

As a boundary case, Mind-of-Director produces an editable 3D film previsualization rather than a finished video. From an initial creative idea, specialized agents iteratively develop the script, construct semantically aligned 3D scenes, plan character blocking and motion, and optimize camera framing and movement, while a real-time visual editor allows creators to inspect the result and adjust the synchronized timeline before production [52].

AnimeAgent

is an I2V-based multi-agent framework for custom storyboard generation. A Director converts the story and visual references into a structured textual dope sheet, while an Artist generates continuous motion trajectories with an image-to-video model. A Consistency Reviewer detects identity, pose, and layout errors and feeds them back for revision, while a mixed subjective–objective reviewer selects the most expressive frames as the final storyboard [53].

5.2. Multi-Shot Production and Cross-Modal Orchestration

Production systems assign directing, cinematography, character and scene management, audio, and generation to different agents. The main distinction is what information passes between them and whether observed results can change later decisions.

AniMaker

is a multi-agent framework for long-form animated storytelling. A Director Agent creates the storyboard, a Photography Agent uses MCTS-Gen to efficiently explore and generate promising clip candidates, and a Reviewer Agent evaluates them with AniEval for story-level consistency, action completion, and animation quality. A Post-Production Agent then assembles the selected clips and adds editing and voiceover [295]. Figure 23 compares this storyboard-driven production path with editable 3D previsualization.

AniME

is a director-oriented multi-agent system for long-form anime production from story to final video. A central Director Agent decomposes the story into dependent production tasks, maintains a shared Asset Memory Bank, and coordinates specialized agents for character and scene design, storyboarding, animation, audio, and editing. Each specialized agent uses an MCP-based toolset to select appropriate models for its task, while quality evaluation can trigger targeted revisions [54].

Cutscene Agent

automates end-to-end 3D cutscene production through an MCP-based Cutscene Toolkit that gives LLM agents bidirectional access to the game engine: agents not only invoke engine operations but continuously observe real-time scene state, enabling closed-loop generation of editable, engine-native cinematic assets [55]. A director agent orchestrates specialist subagents for animation, cinematography, and sound design under a visual reasoning feedback loop, and the accompanying CutsceneBench evaluates long-horizon orchestration of dozens of interdependent, strictly ordered tool calls.

DreamFactory

organizes LLM agents as a virtual film-production team for multi-scene long-video generation. During iterative keyframe creation, a visual Monitor extracts persistent style, background, and character information from generated frames, while contextual information from the current scene is carried into subsequent rounds to guide later keyframes and maintain cross-scene consistency [56].

Kubrick

generates synthetic videos through collaboration among Director, Programmer, and Reviewer agents. The Director decomposes a text description into filmmaking sub-tasks, the Programmer translates them into Blender Python scripts, and the Reviewer inspects intermediate renders and character motion to provide feedback. The Programmer then iteratively revises the scripts based on this feedback before the final video is rendered [9].

Mora

is a multi-agent framework that combines specialized open-source modules to support six video-generation tasks, including text-to-video, image-to-video, video extension, editing, video connection, and digital-world simulation. Its agents are jointly adapted through a self-modulation mechanism, while synthetic data generation and human-assisted filtering provide training supervision. At inference time, these modules are largely composed through predefined task workflows rather than an explicit artifact-conditioned feedback loop [223].

StoryAgent

is a multi-agent framework for generating customized storytelling videos from a narrative and a reference subject. Specialized agents handle story design, storyboard generation, video creation, coordination, and result evaluation, with an Observer providing feedback to the relevant agent when outputs need improvement. A subject-aware storyboard pipeline preserves protagonist identity across shots, while LoRA-BE improves temporal consistency within each shot [296].

AutoMV

generates full-length music videos through a music-aware multi-agent pipeline. Music-processing tools extract song structure, vocals, and time-aligned lyrics; a Screenwriter organizes these cues into a shot-level narrative and shared character bank, while a Director prepares keyframes and video instructions for each shot. A Verifier checks generated candidates for script alignment, character consistency, and physical feasibility, allowing weak outputs to be regenerated or replaced before final assembly [297].

Communicative Agents for Slideshow Storytelling

VGTeam generates slideshow-style narrative videos through a Chat Tower of role-specialized LLM agents. A Director coordinates editor, painter, and composer agents, while shared captions and a Memory Stream carry narrative context across stages. The Director reviews intermediate outputs and returns revision feedback when needed, and image, speech, and music APIs produce the assets assembled into the final video [298].

Hollywood Town

introduces OmniAgent, a hierarchical graph-based multi-agent framework for minute-scale video production. Temporary hypergraph discussions allow selected agents to share task-relevant context on demand, while bounded backward connections let downstream agents request revisions from earlier production stages. This enables cross-stage feedback and iterative refinement without requiring every agent to maintain the full production history [299].

MAViS

is an end-to-end multi-agent framework for long-sequence video storytelling, coordinating specialized agents for script writing, shot design, character modeling, keyframe generation, animation, and audio generation. At each stage, agents follow an Explore–Examine–Enhance procedure to inspect and refine intermediate results, while dedicated script-writing guidelines adapt the narrative to the capabilities of downstream generative tools [300].

MM-StoryAgent

generates narrated storybook videos by coordinating agents across text, image, and audio modalities. A multi-stage writing pipeline first develops the story, after which modality-specific agents iteratively refine prompts for illustrations, narration, music, and sound effects. Character descriptions and consistency-aware image generation help preserve recurring roles, and a composition agent combines the resulting visual and audio assets into the final video [301].

UniVA

is an open-source video generalist that unifies video understanding, segmentation, editing, and generation within a Plan-and-Act architecture. A Planner interprets user intent and decomposes it into structured steps, while executor agents invoke modular MCP tools for analysis and video manipulation. Hierarchical memory preserves global knowledge, task context, and user preferences across multi-step interactions, supporting iterative and self-reflective video workflows [302].

Google Flow

is an AI creative studio whose Flow Agent plans and reasons through creative tasks using user inputs and project context. It supports brainstorming, generation, editing, parallel variation, and batch revision across project assets, while Gemini Omni allows users to iteratively refine videos through conversation [57].

Luma Agents

is a deployed multimodal creative system that coordinates image, video, audio, and editing models within a shared project workspace. Creative agents maintain project context, route tasks across specialized models, and evaluate generated assets before advancing or revising the production process. The available evidence comes from first-party product documentation rather than an independently evaluated technical study [303].

Beyond End-to-End Video Models

LASEV generates educational videos through a hierarchical multi-agent workflow rather than direct end-to-end video synthesis. An Orchestrating Agent coordinates specialized agents for problem solving, executable illustrations, and pedagogical narration, while semantic critiques, rule-based checks, and compilation tests provide iterative quality control. The resulting executable video script is then compiled into synchronized visuals and narration [304].

BrandFusion

is a multi-agent framework for integrating advertiser brands into text-to-video generation while preserving user intent and natural scene composition. An offline phase builds a shared Brand Knowledge Base from model priors and adapts the system to novel brands, while five online agents iteratively refine user prompts using this knowledge and real-time contextual tracking to improve brand visibility, recognizability, and semantic alignment [305].

Camera Artist

is a multi-agent framework for long-form narrative video generation with explicit cinematic language. Its Cinematography Shot Agent recursively plans each shot from the global script, current scene, and preceding shot to maintain shot-to-shot narrative continuity, then enriches the resulting descriptions with professional cinematic language. A Video Generation Agent uses these shot descriptions together with character and scene references to render and assemble the final video [306].

Co-Director

formulates generative video storytelling as hierarchical optimization rather than a fixed chain of prompted modules. A multi-armed bandit explores global creative configurations, while a local multimodal self-refinement loop evaluates generated results and corrects inconsistencies such as identity drift. This combination supports both creative exploration and sequence-level coherence across the resulting narrative [307].

MAVIN

generates synchronized multi-shot audio-visual narratives from free-form prompts and optional character references. A lightweight multi-agent scripting pipeline parses shot structure, aligns character identities with visual and audio references, and refines the result into hierarchical global-, shot-, and role-level captions. Boundary-aware attention enforces temporal alignment across shots and dialogue intervals, while ID-aware propagation binds recurring characters to consistent visual appearances and vocal timbres [308].

Sima 1.0

organizes long-form documentary production as a hybrid human–agent workflow. Human creators retain core creative decisions and physical recording, while junior and senior AI agents handle labor-intensive tasks such as caption refinement, asset collection, split-level editing, B-roll integration, and final asset preparation. During editing, agents can also source additional materials when visual coverage is insufficient, introducing limited runtime adaptation into an otherwise structured production pipeline [309].

ViMax

is an agentic framework for long-form narrative video generation that coordinates specialized agents for screenwriting, shot planning, character styling, video generation, and quality control. Hierarchical story decomposition with retrieval-augmented generation keeps local planning grounded in global narrative context, while a graph-based dependency mechanism tracks character and environment relationships across shots. VLM-based quality control evaluates multiple generated candidates and selects outputs that best preserve visual fidelity and narrative consistency [217].

MAVEN

improves cultural fidelity in text-to-video generation through multi-agent prompt refinement. Specialized agents enrich the person, action, and location dimensions of a prompt either sequentially or independently in parallel, with the parallel variant using an additional fusion agent to combine their outputs. The refined prompt is then passed to a fixed video generator, making MAVEN primarily a prompt-level orchestration framework rather than an artifact-conditioned feedback system [310].

CineAGI

generates multi-scene movies through hierarchical coordination of narrative, character, video, dialogue, and music components. A cinematic blueprint provides global guidance for scene construction, while decoupled character integration supports identity consistency across shots. Hierarchical audio–visual synchronization further aligns dialogue, facial motion, and music throughout the movie [311].

FilmWorld

formulates novel-to-film generation as dynamic cinematic world modeling. Construction agents translate prose into persistent entities, visual anchors, and shot plans, while Evolution agents propagate entity states as scenes are generated. Closed-loop verification checks generated scenes against these evolving states and triggers correction when visual or causal continuity is violated [58].

5.3. Editable Timelines and Post-Production

Timeline-based editors begin with existing footage. They must map language to precise temporal spans, preserve untouched material, and retain enough production history to make later revisions traceable.

LAVE

is an LLM-assisted video editing system that augments existing footage with language descriptions and allows users to edit through natural-language interaction. Given an editing objective, its agent plans and executes operations over the footage, while users can approve, revise, cancel, or directly modify the resulting edits through the interface. This design keeps human control central while lowering the barrier to planning and manipulating video timelines [59].

EditDuet

formulates non-linear B-roll editing as an iterative collaboration between an Editor and a Critic. The Editor searches a supporting footage collection and uses standard editing operations to trim, insert, remove, and rearrange clips on the timeline. The Critic evaluates the current timeline against the user request and either returns natural-language feedback for another editing round or finalizes the sequence for rendering [60].

Prompt-Driven Agentic Video Editing System

This system restructures long-form, story-driven footage through free-form editing prompts. It first constructs a persistent, time-aligned semantic index that combines global narrative summaries with fine-grained scene descriptions and timestamps. Specialized agents then plan the narrative, retrieve and align source clips, and compose narration, subtitles, music, and selected footage into the final edit, while exposing intermediate artifacts for inspection and refinement [312].

Aurora

pairs a tool-using vision–language agent with a unified video diffusion editor to handle underspecified video-editing requests. The agent converts a raw request into a model-ready edit plan, determines the required editing operation, and can retrieve reference images or localize target objects with masks when these conditions are missing. The completed conditions are then passed to the video editor to produce the requested modification [61].

Crayotter

supports traceable long-form video editing through a multi-agent workflow built around persistent production artifacts. It converts retrieved and analyzed footage into a time-grounded editing blueprint and executes the plan through registered timeline tools, while preserving plans, tool actions, intermediate renders, and diagnostics as an observable execution history. When failures occur, the affected segment can be selectively re-executed, and interrupted workflows can resume from validated checkpoints rather than restarting the entire edit [313].

CutClaw

edits hours-long raw footage into short, music-synchronized videos through a coarse-to-fine multi-agent workflow. Hierarchical multimodal decomposition organizes the source video and music into searchable scenes and structural audio units, while a Playwriter Agent anchors the narrative plan to musical sections and keypoints. An Editor Agent then performs fine-grained temporal grounding and trimming, and a Reviewer Agent validates each candidate for semantic relevance, temporal constraints, and visual quality, triggering backtracking and alternative clip selection when a candidate fails [62].

GLANCE

performs music-grounded non-linear video editing through coupled global and local agent loops. An outer loop plans the long-range timeline and decomposes it into segment-level editing tasks, while local agents follow an Observe–Think–Act–Verify process to construct and refine individual subtimelines. When independently edited segments create conflicts after composition, a global–local coordination mechanism identifies the affected regions and uses bottom-up negotiation to revise them while preserving the overall narrative and musical structure [314].

VideoAgent

is an all-in-one framework for long-video understanding and editing. Shot-planning agents and cross-modal retrieval organize source material for coherent narrative construction, while an intent parser selects relevant capabilities from more than thirty specialized editing agents. Textual-gradient graph optimization then assembles these agents into task-specific editing workflows, allowing complex user requests to be executed through dynamically composed pipelines [315].

DIRECT

creates video mashups from existing footage through hierarchical multi-agent planning and intent-guided editing. A Screenwriter anchors the global structure to the source footage and music, a Director translates this structure into segment-level semantic, stylistic, and rhythmic guidance, and an Editor retrieves and dynamically trims shot sequences to optimize local visual and audio coherence. A closed-loop validator evaluates the resulting candidates and, when they fail to satisfy the guidance, prompts the Director to revise the retrieval query before another editing round [63].

5.4. Temporal Feedback, Physical Control, and Local Recovery

Feedback is useful only when it identifies the failed interval and its likely cause, whether narrative, geometric, physical, camera-related, or model-specific. Several systems therefore repair individual shots, prompts, trajectories, or code instead of regenerating the full sequence.

Soap2Soap

addresses long-horizon cinematic video remaking, such as stylization and actor replacement, while preserving the source narrative structure and motion choreography across hundreds of shots. A scene-aware structured screenplay provides a persistent semantic backbone, while dynamically assigned scene- and shot-level visual anchors maintain long-range visual consistency. Batch keyframe generation further reduces drift before video synthesis, and a closed-loop verification agent audits identity, stability, and alignment, selectively regenerating outputs that fail these checks [316].

VISTA

improves text-to-video generation through a test-time self-refinement loop. It first expands a user idea into temporally structured prompt candidates and uses pairwise tournaments to select the strongest video–prompt pair. Specialized agents then critique the selected video along visual, audio, and contextual dimensions, and a Deep Thinking Prompting Agent synthesizes their feedback to revise the prompt before the next generation round. The process repeats until a stopping criterion or iteration limit is reached [64].

Ray3

is an industrial video-generation system presented as evaluating and refining its own outputs during generation. Its reasoning mechanism plans complex scenes with textual and visual representations, judges early drafts, and retries when they do not meet the intended quality. These mechanisms support an artifact-conditioned revision claim, although their effectiveness is documented primarily through first-party technical reports [317].

GenMAC

addresses compositional text-to-video generation through an iterative Design–Generation–Redesign workflow. After each generation round, a sequence of verification, suggestion, correction, and output-structuring agents analyzes the generated video and determines how it should be improved. A self-routing mechanism selects an appropriate correction strategy, and the resulting feedback updates the text prompt, frame-wise layouts, and guidance scales before the next generation attempt [65].

CoAgent

generates coherent multi-shot videos through a plan–synthesize–verify workflow with selective recovery. A Storyboard Planner decomposes the input into shot-level plans with explicit entities, spatial relations, and temporal cues, while a Global Context Manager maintains entity-level context across shots. A Visual Consistency Controller guides shot generation, and a Verifier Agent inspects intermediate results and selectively regenerates shots when inconsistencies are detected. A pacing-aware editor then refines transitions and temporal rhythm for the final sequence [318].

See Before You Code

OmniManim generates executable educational animations through explicit visual planning and render-aware repair. A shared scene state coordinates semantic parsing, layout planning, code generation, and correction, while a Vision Agent predicts sparse keyframe layouts under interpolation-aware constraints to reduce spatial failures between frames. After rendering, structured diagnostics identify overlap, relation, boundary, and execution errors, allowing a Repair Agent to revise the affected code locally rather than regenerate the entire animation [319].

MoReGen

generates physics-grounded videos of Newtonian motion through executable simulation code and iterative feedback. A Text-Parser Agent converts a natural-language description into a structured physical specification, a Code-Writer Agent translates it into executable simulation code, and a Video-Render Agent renders the resulting trajectories. An evaluator then compares trajectories estimated from the rendered video with the simulated motion and assesses physical plausibility and prompt alignment, returning feedback to the Code-Writer for another refinement round. MoReSet complements the framework with annotated trajectories for evaluating physical validity [66].

NEWTON

treats video generation as one action within a learned physics-aware planning loop. A planner orchestrates tools such as keyframe generation, scientific computation, and prompt refinement to construct richer physical conditioning before invoking the video generator. A physical-plausibility verifier then evaluates the generated video and, when the result is unsatisfactory, feeds its assessment back to the planner for another round of tool selection and generation without modifying the underlying video model [7].

FantasyHSI

synthesizes long-horizon human–scene interactions in unseen 3D environments through a graph-based multi-agent framework. A Scene Navigator Agent perceives the environment and plans high-level paths, while a Planning Agent decomposes long-horizon goals into atomic actions. A Critic Agent closes the loop by measuring deviations between generated actions and the planned trajectory and correcting accumulated drift during execution. The action generator is further trained with Direct Preference Optimization to improve physical realism and reduce motion artifacts [320].

CHIEF

supports creator-driven recurrent video refinement through a human-in-the-loop agentic feedback loop. Persona-conditioned multimodal models watch the generated video and provide subjective critiques from different audience perspectives, while the human creator selects and supplements the feedback according to their creative intent. A specialized Refiner Agent then incorporates these revisions into the next generation round, keeping the creator in control of the evolving narrative and visual direction [321].

Closed-Loop Triplet Synergistic Generation

CoTriSyGen formulates long-form video generation as a closed loop among generated visual evidence, text conditions, and entity-centric memory. A vision–language analyzer inspects current keyframes and clips and supports two forms of correction: intra-shot refinement triggers targeted regeneration when local semantic or compositional violations are detected, while inter-shot refinement updates the evolving entity memory and rewrites subsequent shot prompts using newly generated visual evidence. This feedback loop reduces accumulated identity and consistency drift across shots [322].

Genflow Ad Studio

generates brand-aligned advertising videos through a constraint-grounded, self-correcting pipeline. A Brand DNA module extracts explicit corporate constraints such as approved colors, typography, and visual rules, which condition subsequent generation. After each scene is rendered, Director and Brand Safety agents inspect the output for cinematic, brand, and policy violations. When a failure is detected, an Orchestrator converts the critiques into a corrective prompt and regenerates the affected scene until the evaluators accept it or the retry limit is reached [323].

MUSE

formulates long-form audio–visual storytelling as a closed-loop constraint-enforcement process. An omni-modal controller converts narrative intent into persistent identity states and explicit spatial and temporal controls, which guide shot-level visual and audio generation. Generated outputs are then verified for typed violations such as identity mismatch, spatial inconsistency, and temporal discontinuity, allowing the system to revise the affected identity assets, layouts, generation routes, or temporal constraints and selectively regenerate only the problematic content [324].

PhysAgent

generates physically plausible videos through a reflective loop over executable physical control programs. Each program is factorized into a Scene State describing the reconstructed environment and an Event Program specifying forces, trajectories, timing, and interactions through compact physics-control APIs. Stage-specific verification separately checks scene reconstruction and simulated dynamics, and a planner converts detected failures into targeted program edits while preserving validated decisions. Only an accepted simulation is then used as a motion prior for final video synthesis [67]. Figure 24 contrasts this repair process with interface-level and generative revisions.

SCMAPR

performs self-correcting prompt refinement for complex text-to-video scenarios through stage-wise multi-agent collaboration. A Scenario Router assigns the request to a taxonomy-grounded scenario, a Policy Generator constructs a corresponding rewriting strategy, and a Prompt Refiner produces the revised prompt. Semantic verification then decomposes the original request into atomic elements and checks whether they are preserved in the refined prompt; detected omissions or contradictions trigger targeted revision before the prompt is passed to the video generator [68].

5.5. Skill Evolution and Diagnostic Evidence

Two systems go beyond one-off correction by retaining what previous failures reveal. SCPE stores prompting lessons in a Playbook, whereas VideoWeaver refines reusable composition skills across long-video tasks.

SCPE

uses image-to-video generation as a diagnosable process for human–object interaction editing. An Analyzer inspects generated video frames to identify interaction failures, a Reflector abstracts these failures into reusable insights, and a Curator stores them in a Dynamic Playbook that links recurring failure patterns to prompting strategies. The updated Playbook guides subsequent prompt refinement, allowing experience from previous failures to transfer across editing samples, after which the best-aligned video frame is selected as the final edited image [69].

VideoWeaver

studies how general-purpose agent harnesses can construct and improve workflows for long-video generation by composing multimodal foundation skills rather than following predefined pipelines. An agent-as-judge evaluates both execution traces and final videos, using evidence from metadata and intermediate artifacts to diagnose process and output failures. The resulting feedback drives a skill-evolution procedure that progressively refines composition and creator skills and merges them into reusable workflows that can improve performance on later and unseen tasks [70]. Figure 25 relates this cross-task skill evolution to within-trajectory state persistence.

6. Agentic 3D, CAD, and World Generation

Three-dimensional creation makes state more explicit but also makes visual plausibility an insufficient success criterion. Geometry, topology, dimensions, joints, scene graphs, operation histories, collisions, and simulator outcomes can all be inspected, so agents can act on executable structure rather than pixels alone. This section separates appearance-oriented world construction from editable CAD and from verification-driven refinement.

Figure 26. Representative systems and benchmarks for agentic 3D, CAD, and world generation, organized according to the chapter taxonomy.

Figure 26. Representative systems and benchmarks for agentic 3D, CAD, and world generation, organized according to the chapter taxonomy.

6.1. Scene Graphs, Assets, Layouts, and Interactive Worlds

World construction combines asset selection, spatial arrangement, scene-graph state, and sometimes simulation. The key distinction is whether an agent merely writes a scene description or can observe and revise the instantiated world.

Multi-agent Evolutionary Systems for the Generation of Complex Virtual Worlds

This early study combines an Interactive Genetic Algorithm with a learning agent for procedural city generation: a human selects preferred cityscapes while a J48 decision-tree agent observes those choices and incrementally learns to select candidates on the user’s behalf, gradually shifting control to mitigate fatigue [325]. Across 36 runs, the agent-assisted method used more total generations but roughly half the human evaluations compared to the IGA.

Genie

trains an 11B-parameter foundation world model on 30,000 hours of unlabeled Internet gameplay videos: a spatiotemporal video tokenizer, a MaskGIT dynamics model, and a fully unsupervised latent action model together turn a single text, image, or sketch prompt into an action-controllable environment that is stepped through frame by frame [326]. The learned latent action space further lets agents imitate behaviors from action-free videos, though the in-loop decisions remain with the human or downstream agent rather than the model itself.

GameNGen

replaces the hand-written game loop with a neural engine: an RL agent first plays DOOM to record training sessions, then an augmented Stable Diffusion v1.4 learns next-frame prediction conditioned on past frames and actions, with noise augmentation stabilizing long autoregressive rollouts [191]. The engine simulates playable DOOM at 20 FPS on a single TPU and persists game state—health, ammo, doors—over multi-minute trajectories, but the action stream still comes from the player, so the system supplies an interactive world rather than agentic decisions.

Genie 2

scales the generative-interactive-environment paradigm to rich 3D worlds: an autoregressive latent diffusion model with a causal-masked transformer dynamics model turns a single prompt image into a playable environment that responds to keyboard and mouse actions frame by frame, sustaining consistent worlds for up to a minute with emergent physics, object affordances, NPC behavior, and long-horizon memory of off-screen regions [188]. A SIMA agent can be placed inside the generated world to follow natural-language instructions, so the model supplies the interactive environment while task-level decisions remain with the embedded agent or human player.

ShareVerse

targets multi-agent shared world modeling in video generation: four-view videos of independent agents are spatially concatenated to cover a broader environment with internal multi-view geometric consistency, while cross-agent attention blocks injected into a pretrained video model transmit spatiotemporal information across agents so that overlapping regions stay consistent and non-overlapping regions remain plausible [218]. A CARLA-built dataset of diverse scenes, weather conditions, and paired multi-view trajectories supports 49-frame generation that tracks dynamic agents across views.

MultiWorld

scales action-conditioned video world models to multi-agent, multi-view settings: a Multi-Agent Condition Module gives precise per-agent control, a Global State Encoder keeps observations coherent across views, and all views are synthesized in parallel so that agent and view counts scale flexibly [327]. On multi-player game environments and multi-robot manipulation tasks, it improves video fidelity, action-following, and multi-view consistency over baselines.

MetaWorld

scales multi-agent world modeling from single-view videos alone: Monocular World-State Unrolling decomposes monocular footage into camera ego-motion and subject trajectories, extracting synchronized multi-agent motion in a shared 3D space without multi-camera recordings; a Subject-Aware World Generator conditions simulation on per-agent identity images; and World-State Alignment—per-frame inter-branch cross-attention at every DiT layer—synchronizes denoising so that both egocentric views stay grounded in the same physical reality [328].

Prisma-World

formulates multi-agent world generation as a joint geometry-aware denoising process: all agent videos are processed within one full-attention sequence, a multi-agent RoPE distinguishes agent identities under synchronized temporal coordinates, and relative camera geometry injected into attention biases overlapping viewpoints toward shared scene evidence [329]. An overlap-decaying curriculum and minimap-conditioned structural guidance further strengthen cross-view consistency, supported by PrismaDataset, a large-scale UE5 dataset with composable multi-agent view groups and precise camera/action annotations.

OmniDrive

recasts controllable multi-view driving video generation as LLM-choreographed latent choreography: three Qwen2.5-VL agents—a Director parsing user intent into a structured WorldScript, a Cartographer grounding it into spatially anchored layout tokens, and an Auditor feeding cross-view critiques back as auxiliary supervision—jointly author a single position-aware token sequence that is co-compressed with multi-view video through a view-time permutation inside a 3D VAE [330]. On nuScenes it reports state-of-the-art multi-view consistency and BEV mAP, and a detector trained purely on its synthetic data gains +2.4 NDS on the real validation split.

WorldWeaver

augments streaming multi-agent autoregressive video diffusion with world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk, supervised by signals spanning agent status, bird’s-eye views, and scene text, while a Mixture-of-Transformers separates world-state modeling from visual frame modeling [331]. On two-agent Minecraft generation, explicit world-state modeling improves logical consistency and generation quality, giving the world model a persistent state that carries across agents and views.

ShapeCraft

uses a multi-agent system with a Graph-based Procedural Shape representation to generate structured, editable 3D assets: Parser, Coder, and Evaluator agents decompose text into semantic components, iteratively render, score, and refine candidate geometries, with component-aware texture applied post-construction [71]. Figure 27 illustrates this component-level construction process.

Agentic Designer

progressively decomposes an interior layout among agents and applies structure-aware checks after each stage, so invalid spatial arrangements are corrected before detailed rendering [332].

Authoring for Living Worlds

Tool-constrained LLM agents author executable multi-actor event graphs for a 3D world: a Director explores a capability registry and delegates scenes to a Scene Builder that constructs per-actor chains through validated tools against a transactional state backend, emitting only valid specifications by construction [333]. A six-stage GPT-5 pipeline produced zero executables in 50 attempts. The system’s visual output is bounded by the engine’s fixed cameras.

CoGen3D

keeps user intent, design memory, and iterative choices in a human–AI co-design loop for virtual-reality assets rather than automating away creative authority [334].

MA3DSG

addresses the scalability challenge in 3D scene graph generation by distributing object, relation, and scene-graph construction across multiple agents, targeting large indoor environments that exceed the capacity of conventional single-agent systems [335].

One Sentence, One Drama

maintains character, set, and event state while agents plan and critique a personalized short drama, using 3D-grounded first-frame generation to enforce cross-clip spatial consistency [336].

OrchestrXR

uses three specialized agents to progressively transform XR research ideas into structured study, scene, and interaction specifications, with live Unity streaming for grounded user feedback and revision [337]. A user study with 12 researchers shows strong intent preservation (SPS=0.86) and positive usability (SUS=79.17), though design-to-scene grounding remains the main bottleneck.

WorldAgents

uses a multi-agent framework—Director, Generator, and two-stage Verifier—to extract navigable 3D worlds from 2D image foundation models via iterative inpainting and 3D-consistent verification [74]. The system relies on VLM judgments, inherits geometric drift, and follows a fixed left/right exploration policy.

Idea23D

generates 3D models from interleaved multimodal inputs (text, images, and 3D models) through three LMM-based agents for prompt generation, model selection, and feedback refinement. The Prompt Generator converts multimodal IDEAs into text prompts; the Model Selector picks the best 3D candidate via multiview rendering; the Feedback Refiner analyzes gaps and suggests improvements. A memory module stores history to guide iterations. [72].

MuMA

decouples PBR texturing into multi-view shaded and albedo generation, then leverages an intrinsic decomposition model to recover metallic and roughness, with a VLM agent scoring and selecting between generated and decomposed albedo candidates [338].

RAISECity

retrieves real-world geographic and visual evidence, uses multimodal agents to imagine complete building appearances from incomplete observations, generates city-scale 3D content aligned to OSM layouts, and applies iterative VLM-based reflection and refinement to correct geometric and semantic deviations between generated worlds and their real references [339].

WorldCraft

uses LLM agents to plan assets via a self-growing manual that masters procedural generators, and to arrange layouts via hierarchical numerical optimization with ergonomic and aesthetic constraints [73].

A Multi-Agent Framework for Democratizing XR Content Creation in K–12 Classrooms

assigns ideation and content-construction roles to agents while keeping teachers and students in control, lowering the tooling barrier to executable classroom experiences [340].

Articraft

generates articulated assets through executable code and stores generation traces for future fine-tuning, providing a scalable path from static 3D shapes to movable structures [341].

Metric-Guided Synthetic Rendering

generates synthetic image data via procedural rendering and modulates scene parameters under task metrics, making downstream learning utility part of the objective [342].

NaLA

is a 3D-native layout agent that takes point-cloud geometry as input and predicts poses directly, avoiding the loss introduced when spatial decisions are mediated through 2D renderings or text [343].

PhysCodeBench

provides a benchmark for physics-aware symbolic simulation, while SMRF uses self-corrective multi-agent refinement to repair violated physical constraints [86].

SceneConductor

constructs 3D scenes from a single image via geometry-aware layout prediction and environmental scaffolding, then refines the scene through planner-directed simple fixes and specialist-isolated complex corrections [344].

SimWorlds

plans assets and dynamics through a staged construction pipeline, then verifies each stage against Blender engine state—modifier stacks, physics caches, and collision relations—rather than rendered images alone, enabling mechanism-correct 4D scene generation that visual-only critics cannot ensure [75].

Vinedresser3D

decomposes text-guided 3D edits into localized asset operations, supporting precise modification of existing scenes rather than full regeneration [345].

AGILE: Hand–Object Interaction Reconstruction from Video via Agentic Generation

reconstructs simulation-ready hand-object interaction assets from monocular video via VLM-guided generation and contact-aware optimization, producing watertight meshes and 6D pose trajectories rather than visually plausible renderings [346].

6.2. Executable CAD Programs and Editable Solids

CAD makes action history and constraints first-class. Code, sketches, feature operations, and parametric edits provide precise interventions, while rendered views offer complementary visual evidence that the underlying solid remains meaningful.

CrossMatAgent

automates metamaterial design via a four-agent team (Describer, Architect, Builder, Supervisor) that analyzes input patterns and iteratively refines DALL-E 3 prompts through supervisor-guided reflection. The resulting 700 image-text pairs fine-tune SDXL with CLIP alignment, generating 2D patterns that are binarized and path-connected by design—not post-hoc—enabling direct use in simulation meshing and 3D printing slicers without heuristic repair, though outputs remain raster patterns rather than parametric CAD or editable solids [347].

From Idea to CAD

converts sketches, photos, and text into parameterized CadQuery models via a three-agent team—Requirements Engineer, CAD Engineer, and Quality Assurance Engineer—that iteratively clarifies specifications, generates and repairs code with compiler feedback, and verifies outputs against multi-view renderings, with user confirmation as an outer validation loop [76].

Agent-Aided Design for Dynamic CAD Models

targets the "scissors test"—generating 3D assemblies with moving parts and revolute joints—by having an agent write JSON definitions for parts, links, and degrees-of-freedom, compiled through a modified multibody constraint solver with quaternion-based orientation and FreeCAD rendering. Unlike prior static CAD generation, the agent iteratively repairs joint definitions using solver errors and uniquely colored per-instance renderings that disambiguate identical components. [348].

ArtiCAD

generates editable, articulated CAD assemblies from text or images via a four-agent system that introduces Connectors—explicit attachment points with joint types and local frames—specified at design time rather than inferred during assembly, reducing connection matching to deterministic alignment. Per-part scripts are generated with local validation; assembly occurs without LLM calls; cross-stage rollback isolates design versus code errors. [77]

Clarify Before You Draw

ProCAD decomposes text-to-CAD into proactive clarification followed by code synthesis, addressing prompts with missing or conflicting dimensions via a clarification agent that asks targeted questions in a single turn before passing a standardized specification to a fine-tuned CadQuery coder [349]. On ambiguous prompts, ProCAD substantially outperforms direct Claude Sonnet 4.5 in Chamfer distance and invalidity; human evaluation on 100 examples confirms the trend.

Zero-to-CAD

synthesizes executable, editable CAD construction sequences at scale without real design histories, using an LLM with tool access that iteratively generates code, validates geometry, and consults documentation to repair errors [350]. The pipeline produces nearly one million sequences covering Booleans, fillets, chamfers, shells, and sweeps, with fine-tuned models demonstrating strong generalization to human-designed CAD and outperforming frontier VLMs on reconstruction tasks.

CAD-Assistant

is a training-free, general-purpose CAD agent that augments a VLLM with FreeCAD tools—execution, rendering, constraint checking, and cross-section extraction—to iteratively plan and revise open-ended design tasks from sketches, scans, or text, moving beyond dataset-bound or task-specific generation [78]. Figure 28 summarizes the task coverage supported through this executable interface.

CADReasoner

frames scan-to-CAD as iterative self-editing: a single model compares rendered geometry with the target via multi-view overlays and point-cloud offsets, then rewrites its own CadQuery program to close the gap, learning to correct from its own prediction errors without RL or frozen VLMs [79].

IterCAD

unifies generation and editing as a closed-loop agent that parses dimensioned engineering drawings and refines CadQuery code using sandbox feedback—compiler logs and OCCT-projected views with explicit dimensions—to localize errors to specific sketch entities rather than global shape discrepancies. Progressive SFT and GSPO with Geometry-Viable Prefix Masking instill self-correction without penalizing early correct turns for downstream failures [80].

6.3. Geometry-Aware Verification, Simulation, and Learning

Verification can inspect geometry, physics, compilation, and task function independently of appearance. These signals also make 3D a useful setting for reinforcement learning and self-corrective search.

CAD-Judge

replaces VLM-based rendering and ranking with a local CAD compiler that reconstructs B-reps or meshes and supplies Chamfer-distance and compilation signals as fast, rule-based rewards during preference alignment, while a Compiler-as-a-Review module appends syntax, closure, extrusion, or Boolean errors to the prompt for iterative regeneration at inference [81].

Physics-in-the-Loop

embeds FEA as a deterministic verification signal into an agentic CAD loop, where a multi-agent system iteratively generates and refines designs until they pass geometry and physics checks for load-bearing validity, rather than optimizing geometric similarity alone [82]. Figure 29 shows how geometric and finite-element evidence returns to planning and code generation.

TOOLCAD

trains open-source LLMs as tool-using CAD agents via online reinforcement learning, where a ReAct-style policy interacts with FreeCAD through MCP-based primitives and receives step-level engine feedback (exceptions, constraint warnings, success/failure) plus trajectory-level outcome rewards from a trained ORM, updated with GRPO across a part-wise curriculum [83].

3D Furniture Layout RL

decompose 3D furniture layout into two coupled 2D control problems (x-y and y-z planes) and train cooperative DQN agents to move furniture under IoU-based geometric rewards, producing 3D layouts that outperform prior 2D-only methods on indoor scene benchmarks [351].

CADSmith

combines RAG-augmented code generation with programmatic geometric validation via OpenCASCADE kernel metrics (volume, bounding box dimensions, face counts, solid validity) and an independent VLM Judge with three-view renders, triggering iterative refinement through nested execution and geometry correction loops [84].

CLARE

proactively gates 3D tool execution by detecting ambiguity, missing information, and mistaken details, then learns clarification policies through simulated user interactions and Multi-turn Reward optimization to balance task success against interaction [85].

3DCodeBench

provides a benchmark and human-preference arena for evaluating VLM agents on procedural 3D modeling via Blender Python code, measuring executability, multi-view perceptual similarity, and 3D geometric alignment across 212 categories, while multi-turn error feedback reveals that API fixes lift executability but do not improve shape fidelity [87].

Three-dimensional domains offer unusually strong heterogeneous evidence, yet the evidence is split across rendered appearance, geometric validity, editability, and physical function. A convincing agent must preserve all four views of state and repair the underlying representation rather than optimizing only a camera projection.

7. Agentic Scientific Figure and Visualization Generation

Scientific and data-grounded graphics bind visual communication to source fidelity. Here the artifact is simultaneously an image, a structured specification, an executable program, and an argument about data. Agentic control must therefore coordinate analytical intent, chart or diagram structure, code execution, visual inspection, and checks that the rendered claim remains supported by the source.

Figure 30. Representative systems and benchmarks for agentic scientific figure and visualization generation, organized according to the chapter taxonomy.

Figure 30. Representative systems and benchmarks for agentic scientific figure and visualization generation, organized according to the chapter taxonomy.

7.1. Data-to-Chart Analysis and Executable Visualization

Chart agents translate analytical intent into data transformations, encodings, and executable plotting code. Execution catches syntax failures, but factual and perceptual validity require additional checks against the data and communicative goal.

LightVA

presents an LLM-agent framework that automates task planning, code generation, and linked visualization creation for exploratory visual analytics from raw datasets and high-level user goals [88]. The framework (Planner, Executor, and Controller) supports human oversight through direct manipulation and natural language. Figure 31 illustrates the visible analytical state and direct user controls exposed by the system.

A2P-Vis

introduces a two-stage multi-agent pipeline that turns raw tabular data into a publication-ready data-visualization report containing executable charts, ranked insights, and narrative text [352]. The pipeline (Data Analyzer and Presenter) demonstrates end-to-end integration through a worked example without controlled evaluation of output quality.

MultiVis-Agent

presents a logic rule-enhanced multi-agent framework that extends text-to-chart generation to four scenarios—basic, image-referenced, code-referenced, and iterative refinement—with mathematical reliability guarantees [89].

ChatVis

is an LLM-based assistant that generates ParaView Python scripts for 3D and time-varying scientific visualization from natural language descriptions [90]. It combines chain-of-thought prompt decomposition, retrieval-augmented generation from a vector database of ParaView documentation and code examples, and iterative error correction that feeds execution failures back to the LLM until the code runs. Empirical evaluation on a benchmark of canonical tasks, regression tests, and scientific use cases demonstrates substantial improvements over unassisted state-of-the-art LLMs.

WaitGPT

is a prototype system that transforms LLM-generated data analysis code into an interactive flow diagram in real time, enabling users to monitor and steer conversational data analysis without deciphering raw code [353]. It parses scripts into visual operation chains with runtime table states and supports node-based inspection and refinement. User studies demonstrate reduced cognitive load and improved error detection over code-only interfaces.

Does It Run and Is That Enough?

This study re-examines text-to-chart generation, questioning whether execution success alone suffices as evidence of chart quality and accessibility [354]. A lightweight two-agent pipeline with interpreter-feedback-driven repair substantially reduces runtime errors on standard benchmarks, yet accessibility analysis reveals that most generated charts fail basic colorblindness criteria and manual review identifies hallucinations even in executable outputs. The paper argues that execution is largely solved and calls for shifting evaluation focus toward semantic fidelity, visual clarity, and accessibility.

METAL

introduces a vision-language model based multi-agent framework that generates chart code from reference images through iterative collaboration among generation, visual critique, code critique, and revision agents [222]. The visual critic diagnoses appearance discrepancies between rendered and reference charts, while the code critic identifies implementation defects; their feedback drives iterative refinement until a quality threshold or compute budget is met.

PlotEdit

introduces a multi-agent framework for natural-language-driven editing of chart images in PDFs or scanned documents where source data and rendering code are unavailable [91]. Five specialized agents extract data tables, visual attributes, and code from the input chart, decompose user editing requests into executable steps, and implement modifications, coordinated through numeric, visual, and code feedback signals that iteratively rectify de-rendering errors. A perceptual fidelity feedback mechanism further ensures that only user-specified changes are applied while preserving unchanged chart components.

Exploring Agentic Workflows for Generating High Quality Math Visual Aids

This study explores an agentic workflow for generating TikZ-based K-12 math diagrams through a QA-driven self-improvement loop [355]. A Diagram Generator produces initial TikZ code, a QA Generator formulates open-ended questions about accuracy and clarity, a QA Evaluator answers them using both code and rendered image, and a Verifier converts failed items into feedback for regeneration.

GA-VisAgent

is a multi-agent framework that translates geometric algebra formulas into executable GAALOPScript code and interactive 3D visualizations for educational purposes [92]. A ReAct-based Planner decomposes input formulas into categorized subtasks, which are routed to specialist agents for code generation, assignment, and visualization, while a validation agent checks syntax and triggers regeneration on failure.

Toward AI VIS Co-Scientists

This work presents an end-to-end agentic harness that autonomously designs interactive scientific-visualization applications (VIS Apps) from raw datasets and high-level research tasks, producing linked views, filters, and analytical panels rather than static charts [356]. A primary code agent orchestrates exploratory data analysis, environment setup, planning, visualization implementation, and Playwright-based browser validation; evaluation reports drive iterative repair until blocking conditions are resolved, with project artifacts retained for traceability.

7.2. Scientific Figures and Diagrams

add domain semantics, symbolic structure, and source-document grounding. Their state must remain editable and traceable to evidence, not merely visually similar to a target style.

From Pixels to Paths

VisPainter is a multi-agent framework that converts natural language instructions into editable vector scientific diagrams by directly operating standard graphics software, keeping modules, connectors, and labels independently modifiable after generation [93]. A Manager orchestrates the workflow, a Designer proposes layouts and refines them via screenshot feedback, and a Toolbox executes over thirty editing operations. The accompanying VisBench benchmark provides 360 high-information-density diagrams with seven-dimensional metrics spanning content, layout, readability, and interaction cost.

SciFig

is a multi-agent framework that generates visually rich, fully editable methodology figures from scientific text, with four agents handling planning, layout, component rendering, and iterative refinement [94]. The output XML preserves individual blocks, arrows, and labels for post-hoc editing in standard tools. And SciFig-Bench provides 435 human-verified methodology figures across 37 AI/ML domains.

EduVisAgent

is a multi-agent framework that generates pedagogical visual explanations for K-12 STEM problems, producing diagrams or interactive web pages that expose reasoning sequences and connect textual steps with visual elements [95]. A planning agent coordinates five specialized roles—conceptual mapping, reasoning decomposition, learning guidance, metacognitive prompting, and visualization design—to construct a pedagogical plan for rendering. EduVisBench provides 1,154 STEM questions across three subjects and 15 domains.

See it. Say it. Sorted

This system converts hand-drawn sketches into editable SVG flow diagrams through an iterative critic–candidates–judge loop [357]. A Critic VLM compares the sketch with the current SVG and suggests qualitative relational edits; multiple LLMs generate diverse candidate SVG modifications; a Judge VLM selects the best candidate, reverting to the prior version if none improves and feeding back the failure.

Crafter

is a multi-agent harness that generates scientific figures across academic, poster, and infographic types from diverse inputs, and converts raster outputs into editable SVGs via CRAFTEDITOR [358]. A designer proposes candidate plans, an executor renders them, a critic diagnoses errors across six quality dimensions, and a reviser applies structured edits to a shared specification, with rollback to the best version when modifications degrade quality. CRAFTBENCH provides 279 samples across three figure types and four input conditions.

Feynman

is an LLM agent that generates knowledge-infused scientific diagrams at scale by decoupling knowledge elicitation from visual production [96]. It enumerates domain-specific ideas, plans Penrose programs, compiles them, and iteratively refines outputs via a panel of visual judges until quality thresholds are met; Penrose’s optimization-based rendering ensures visual diversity while preserving semantics. And the Diagramma benchmark provides 1,058 questions across six subjects.

SASAV

is a fully autonomous multi-agent system that generates scientific visualizations from volumetric data without prior knowledge or human-in-the-loop feedback [97]. It profiles input data, retrieves domain knowledge via web search and RAG, recommends transfer functions through parallel MLLM-based evaluation of isosurfaces, and selects optimal viewpoints and exploratory trajectories—producing static images, animations, and interactive visualizations. The system operates as a fixed four-stage pipeline with batched MLLM inference.

7.3. Rendered Feedback, Semantic Validation, and Benchmarks

Evaluation in this domain can combine inverse parsing, code execution, rule checking, visual critique, and source consistency. The following work shows both the strength of heterogeneous evidence and the remaining difficulty of converting a diagnosis into a correct local repair.

AMACE

iteratively refines table-to-chart generation through a multi-agent loop without manual prompt engineering [98]. A Code Generator produces Python code, a Renderer executes it, a multimodal Replier answers queries from the chart, and an Evaluator checks correctness against the table—feeding feedback back to the generator. An Early Stopping Checker terminates when quality criteria are met.

MAGMA-Edu

is a multi-agent framework that generates K-12 math problems with coherent text-diagram pairs through a two-stage generate–validate–reflect pipeline [359]. Stage 1 refines textual content via Generator, Validator, and Reflector agents; Stage 2 translates verified descriptions into executable Matplotlib code, rendered and iteratively corrected by Code Executor, Image Validator, and Image Reflector agents. The code-based intermediate representation ensures geometric fidelity and deterministic re-rendering after errors.

PlotGen

is a multi-agent framework that generates scientific charts from natural language requests and data through iterative multimodal feedback [99]. A Query Planning Agent decomposes user requests, a Code Generation Agent produces Python code, and Numeric, Lexical, and Visual Feedback Agents inspect the rendered chart—verifying data trends, textual labels, and visual aesthetics—and return diagnoses to revise the code.

HiLSVA

is a human-in-the-loop agentic system that supports mixed-initiative scientific visualization workflows in ParaView through plan-first execution, stepwise provenance tracking, and test-time adaptation [100]. An orchestrator coordinates specialized agents; users can co-plan, approve actions, directly manipulate visualizations, and restore prior states, while a self-improving agent reflects on outcomes, queries users when uncertain, and stores validated knowledge for reuse.

Perceptual Self-Reflection

This framework generates physics simulation code from natural language and validates it through perceptual self-reflection: rendering the animation, sampling frames, and using a vision-language model to check physical criteria—routing diagnoses to requirement or code revision [360]. Iteration continues until quality thresholds are met or ten attempts are exhausted.

Progressive Text-to-Visualization

This paper introduces a progressive multi-turn text-to-visualization paradigm where users iteratively refine queries and each turn yields executable VQL, along with PMVisBench for evaluation and PMVisAgent for execution [101]. PMVisBench is constructed by reverse-simplifying complete VQLs with constraints to ensure intermediate validity. PMVisAgent employs three agents with ReAct-style validation tools to detect and repair errors before propagating to the next round, significantly outperforming one-shot baselines.

Agentic Visualization

This paper presents 11 design patterns for agentic visualization—systems that integrate autonomous agents while preserving human analytical control—extracted from prior visualization literature [361]. The patterns span Agent Roles (Forager, Analyst, Chart Creator, Storyteller), Communication (Insight Timeline, Progress Indicator, Provenance Log), and Coordination (Scouting, Swarming, Monitoring, Consolidating), each specifying problem, solution, examples, and tradeoffs. The catalog is validated through examples and a composite scenario, providing a design vocabulary for future agentic visualization systems.

An Evaluation-Centric Paradigm for Scientific Visualization Agents

This position paper advocates for systematic evaluation benchmarks for SciVis agents, proposing a dual framework of outcome-based and process-based evaluation organized around accuracy, coverage, and cost-effectiveness [362]. Outcome-based assessment uses MLLM judges for final visualization quality; process-based assessment examines intermediate actions via hard-coded state verifiers, token usage, and runtime.

Exploring LLM Agent Designs and Interaction Modalities for Scientific Visualization

This study compares domain-specific, general-purpose coding, and GUI-based agents on 15 ParaView tasks [363]. General-purpose coding agents achieve highest completion but at higher cost; domain-specific agents are more efficient yet less flexible. Persistent memory improves performance, and stepwise decomposition raises GUI agent pass rates to 60-65%, indicating multi-step planning as the primary bottleneck.

SciFlow-Bench

evaluates scientific diagram generation by inverse-parsing pixel outputs into graphs, measuring structural recoverability rather than visual similarity [103]. A multi-agent pipeline constructs canonical graphs from source figures and reverse-parses generated images into predicted graphs for comparison. The benchmark contains 500 diagrams across five domains; among evaluated generators, Gemini 3 Pro Image achieves the highest aggregate score, yet substantial structural loss remains.

SciVisAgentBench

is a reproducible benchmark for evaluating LLM-based agents on scientific visualization tasks, grounded in a taxonomy of domains, data types, complexity, and operations [104]. It comprises 108 expert-authored cases evaluated through an outcome-centric pipeline combining LLM judgment with deterministic checks.A validity study with 12 experts shows Claude Opus 4.6 correlates at 0.806 with human judgments, supporting scalable assessment.

SciVisAgentSkills

is a collection of reusable procedural skills that equip general-purpose coding agents with tool-specific knowledge for ParaView, napari, VMD, and TTK workflows [102]. Each skill encodes environment assumptions, API patterns, examples, and heuristics as a static instruction package loaded at inference time without model retraining. Evaluated on SciVisAgentBench, skills improve mean task scores across most suites—e.g., Claude Code’s ParaView score rises from 62.57 to 73.93—though effects vary by agent and task, and token usage does not consistently decrease.

Executable code and structured specifications make scientific graphics comparatively auditable, but not automatically truthful. Reliable control requires provenance from source to transformation to mark, plus visual and semantic checks that can identify the responsible stage when the final claim is wrong.

8. Agentic Structured Visual Document and Presentation Generation

Structured visual documents couple long-range narrative decisions with page-level geometry. Their state includes source material, section hierarchy, slide or page roles, assets, layout constraints, style, citations, and edit history. The relevant systems are compared by whether they keep these representations connected when content is retrieved, composed, rendered, inspected, and revised.

Figure 32. Representative systems and benchmarks for agentic structured visual document and presentation generation, organized according to the chapter taxonomy.

Figure 32. Representative systems and benchmarks for agentic structured visual document and presentation generation, organized according to the chapter taxonomy.

8.1. Interleaved Reports and Structured Multimodal Content

Reports and interleaved content require the agent to decide when prose, images, tables, or diagrams should carry the argument, and to keep citations and retrieved evidence aligned with those decisions.

A Composable Agentic System for Automated Visual Data Reporting

This auditable multi-agent system automates visual reporting by externalizing logic to deterministic modules, producing interactive Observable reports and executable Marimo notebooks. [105]. The DSPy-driven system delegates chart design to Draco and uses Langfuse for traceability, and while it won the VISxGenAI Challenge, evidence remains case-based.

TVIR

presents a hierarchical multi-agent framework for text-visual interleaved report generation with provenance-tracked visual assets, alongside a 100-task benchmark evaluating citation grounding and multimodal visual quality [106]. Source-grounded chart construction proves central to performance, yet feedback-driven re-planning is absent and its LLM-based evaluation demands broader human validation.

8.2. Presentation and Poster Composition

Slides and posters require coordinated decisions about narrative role, information density, asset selection, typography, and page geometry. Treating each page as an isolated image loses the state needed for coherent revision.

PosterGen

introduces an aesthetic-aware multi-agent framework for paper-to-poster generation, where specialized agents collaboratively distill narrative, allocate layout, and apply color and typography to produce editable PPTX posters [107]. Evaluations show quality approaching human-designed posters, yet the pipeline remains a predetermined cascade without feedback-driven replanning, and its fixed three-column layout defines the boundary.

PreGenie

employs a multi-agent framework for visual presentation generation, where a summarizer and image captioner extract multimodal content, a code generator produces Slidev Markdown, and code and page reviewers iteratively inspect both syntax and rendered visuals to detect overflow, misalignment, and density issues before regenerating problematic pages [108]. Figure 33 presents representative localized repairs triggered by rendered-page feedback.

PresentAgent

transforms long-form documents into narrated presentation videos through a modular pipeline that segments the document, plans and renders slide-style visual frames, generates contextual narration with LLM and text-to-speech models, and composes the final video with precise audio–visual alignment [109]. Its PresentEval framework scores videos on content fidelity, visual clarity, and audience comprehension via VLM-based evaluation, reporting near-human quality on 30 document–presentation pairs, though the pipeline itself remains a fixed cascade without render-driven revision.

ETPDesigner

introduces a multi-agent framework for transforming scripts into interactive theater programs, using RAG-enhanced ToT poster synthesis with VLM self-refinement and a global style anchor for cross-page consistency [110].

Towards Verifiable Multimodal Deep Research

PTAH presents a multi-agent harness for verifiable multimodal deep research, organizing Planning, Research, and Writing phases with a Visual Working Memory that retains image provenance and context, while a Verifier enforces stage-wise factual and cross-modal consistency through rule-based and model-based checks [111]. Test-time scaling with six refinement operations further improves sections, images, and rendered HTML.

8.3. Structured Diagrams and Software Documentation

Software diagrams are visual documents whose correctness depends on named entities, typed relations, and consistency with an underlying repository or requirements specification. These systems are grouped here because their primary artifact is a structured documentation view rather than an analysis of scientific data.

NOMAD

decomposes UML class diagram generation into specialized agents for entity extraction, relationship comprehension, model integration, and PlantUML articulation, with an optional verifier that checks outputs against requirements and requests revisions [112]. Experiments show improved F1 scores (0.656→0.696 on eight cases, 0.736→0.836 on Northwind), yet attribute extraction remains the primary weakness.

Code2UML

employs a multi-agent architecture with deterministic IR compaction to generate seven UML diagram types from source code repositories across four languages, where Planner, Analyzer, Diagram, and Corrector agents collaborate to produce PlantUML outputs with diagram-specific syntax validation [113].

SAGE

is a browser-based tool for prompt-guided editing of software diagrams, parsing Draw.io XML into an editable graph model, translating natural language requests into structured edit intents, and applying deterministic graph operations with validation and versioned rollback [114].

8.4. Rendering-Grounded Revision and Document-Level Evidence

Rendered pages expose overflow, overlap, alignment, and readability failures that are invisible in an outline or source tree. Document-level feedback additionally tests cross-page consistency and the cost imposed on human reviewers.

Multi-Agent Synergy-Driven Iterative Visual Narrative Synthesis

RCPS introduces a multi-stage framework for automated presentation generation, combining R-CoT narrative planning, a learned layout prototype generator that produces symbolic layout descriptions, and an iterative multi-modal refinement loop where visual and logical critics guide targeted edits until quality thresholds are met [115].

Presenting a Paper is an Art

EvoPresent introduces a self-improving multi-agent framework for academic presentation generation, where Storyline, Scholar, Design, and Checker agents iteratively refine slides guided by PresAesth, a multi-task RL aesthetic model trained on scoring, defect adjustment, and comparison tasks [116].

DeepPresenter

introduces a dual-agent framework where Researcher and Presenter collaborate in a shared file environment, using inspect tools to render slides into pixels and expose post-render defects, enabling observation-driven revision [117]. The compact DeepPresenter-9B is trained on filtered trajectories with extrinsic verification, yet inference lacks an external critic, and the system remains sensitive to context overflow and does not support cross-task updating.

ATP-Bench

is a benchmark for evaluating multimodal models’ tool-planning ability in interleaved text-image generation, where models must decide when and how to invoke reference, diffusion, search, code, or edit tools across 25 visual-critical intents and 7,702 QA pairs [118]. A Multi-Agent MLLM-as-a-Judge system separates precision inspection, missed-opportunity detection, and chief judgment, correlating strongly with human evaluations ($\rho =0.89$). Gemini 3 Pro achieves the best reported scores (79.88 final score, 81.77% success rate, 0.49 missed images). As a benchmark, ATP-Bench does not address persistent state, rollback, recovery, or cross-task learning in deployed systems.

DATAREEL

benchmarks automated data-driven video story generation: given a data table, a communicative intent, a target duration, and a style reference image, a model must produce executable animation code with synchronized subtitles, which is rendered and evaluated across 328 real-world data reels [119]. Eight proprietary and open-weight VLMs reveal a substantial capability gap—open-weight models reach up to 39.8% execution failure, and even the best models produce static charts, subtitle–animation desynchronization, and unstable layouts—while an agentic baseline decomposing generation into planning, coding, and verification consistently outperforms direct prompting.

Document agents become genuinely artifact-grounded when outline state, evidence state, layout state, and rendered observations remain synchronized. Current systems establish useful loops around rendering and revision, while source fidelity, cross-page memory, and human approval remain essential controls.

9. Agentic UI and Web Generation

Interface generation joins visual appearance to executable behavior. Component trees, style rules, screenshots, browser state, and interaction traces provide complementary views of the same artifact, making this domain a stringent test of whether visual feedback can be translated into code-level repair.

Figure 34. Representative systems and benchmarks for agentic UI and Web generation, organized according to the chapter taxonomy.

Figure 34. Representative systems and benchmarks for agentic UI and Web generation, organized according to the chapter taxonomy.

9.1. From Visual Specification to Executable Interface

Visual-to-code and specification-to-interface systems must coordinate design intent with component structure and runtime behavior. A rendered screenshot is an observation of the program, not a substitute for functional state.

MM-WebAgent

is a hierarchical multimodal framework that generates coherent webpages with images, videos, and charts from natural language. It operates through global layout planning, local element planning, parallel AIGC-based generation, and three-tier iterative refinement (local asset, surrounding code, and full page) guided by rendered screenshots [120]. Figure 35 presents representative outputs across several multimodal webpage tasks.

MAxPrototyper

is a multi-agent system that converts a text description and a wireframe into an editable UI prototype. A Theme Design Agent sets the global theme and generates a theme image via fine-tuned diffusion with ControlNet, while text, image, and icon agents create each component under its direction, using a cache pool to maintain design context. [121].

GameUIAgent

is an LLM-powered framework that generates editable Figma game UI designs from natural language via a Design Spec JSON intermediate representation, combining LLM generation, deterministic post-processing, and a VLM-driven Reflection Controller that iteratively refines designs with guaranteed non-regressive quality [122]. The Quality Ceiling Effect (r = -0.96) and Rendering-Evaluation Fidelity Principle further reveal that improvement is bounded by evaluator headroom and that incomplete rendering can paradoxically degrade VLM scores.

9.2. Browser-Grounded Verification and Repair

Browser-grounded systems can combine screenshots, DOM structure, interaction tests, and accessibility checks. This heterogeneous evidence makes precise repair possible but also reveals failures that a single image metric cannot capture.

Replit Agent

is a commercial software-building agent that turns a natural-language application description into a deployed web application: it plans the project as an ordered task list, configures the environment, installs dependencies, writes code, and executes the result, testing its own work and fixing detected problems, while checkpoints preserve intermediate states for rollback [123]. Task-level intent and acceptance remain with the user through chat, but the build–execute–repair loop is driven by runtime evidence rather than by re-prompting.

Bolt.new

runs full-stack application generation entirely in the browser: the model is given complete control over a WebContainers-based environment—filesystem, Node.js server, package manager, terminal, and browser console—so that prompting, running, editing, and deploying happen within a single session [124]. Runtime instrumentation captures execution errors and returns them to the model, which automatically tests, refactors, and iterates, though evidence for this loop remains vendor-reported rather than independently benchmarked.

Lovable

is a chat-driven web-application builder whose agent observes build errors, scans project logs and console output, and tests the generated application in a browser to attempt repairs [125]. A scheduled monitoring mode extends the loop beyond user-initiated turns by watching the deployed project and proposing fixes when issues are detected.

PlayCoder

is a multi-agent framework that generates executable GUI application code through a closed-loop of repository-aware generation, automated visual testing, and iterative repair, targeting silent behavioral failures that evade compilation and unit tests [126].

TDDev

is a test-driven development framework that generates full-stack web applications from natural language and optional design images via three agents: test generation, development, and testing (with BrowserUse for interaction simulation), iteratively refining code until all test cases pass [127].

VISTA

is a benchmark evaluating end-to-end web-app coding agents under five input conditions varying visual/structural fidelity and stack constraint, with 3,253 annotated interactions across 128 pages [128]. It combines DOM-grounded behavior checks with CLIP-based visual similarity. Among four model–harness systems, free-stack conditions yield the best Combined scores ( 0.26). Opus achieves the highest Combined (0.261) and behavior (0.336), while GPT-5.5 attains the best CLIP (0.853) but lowest behavior (0.283), revealing a decoupling between visual reproduction and functional correctness.

Vision2Web

is a hierarchical benchmark for evaluating multimodal coding agents on visual website development across three levels—static, interactive frontend, and full-stack—with 193 tasks, 918 prototypes, and 1,255 test cases [129]. It employs workflow-based verification: a GUI agent checks functionality and a VLM judge assesses visual fidelity. Claude Opus 4.5 performs best overall under OpenHands (full-stack VS/FS: 38.4/57.6), while Gemini-3-Pro-Preview leads on static desktop (63.3) but collapses on full-stack (VS 11.7, FS 22.6). The GUI verifier achieves 87.2% node-level agreement with human annotations, and the VLM judge attains a mean Spearman correlation of 0.66.

UI and Web work provides the clearest opportunity to connect perception to executable repair because the same artifact can be inspected as pixels, a component tree, and runtime behavior. The principal challenge is to preserve this correspondence across long editing trajectories rather than optimizing isolated screenshots.

0.80

10. Cross-Domain Agentic Visual Generation

This chapter examines agentic visual generation across domains whose artifacts, interfaces, and evaluation signals differ. The comparison follows the operating loop introduced earlier: the retained state identifies what the system can address, the selected action changes an artifact or execution process, and the available evidence determines the next operation. The cases cover multimodal storytelling, long-horizon multimodal interaction, game-development evaluation, node-based editing, embodied navigation videos, personalized training media, and video-generation infrastructure. They show how the same control questions are instantiated when the system coordinates several media, exposes an editable graph, uses a visual trajectory to guide physical action, checks human constraints, or optimizes deployment.

Figure 36. Representative systems and benchmarks for cross-domain agentic visual generation.

Figure 36. Representative systems and benchmarks for cross-domain agentic visual generation.

Multimodal Qinqiang opera generation.

A multi-agent Qinqiang opera framework converts a source story into a culturally grounded script, stage-scene descriptions and images, and synthesized vocal performance. One agent generates acts, scenes, dialogue, stage directions, and performance cues; a second converts script descriptions into visual scenarios; and a third assigns speech-synthesis parameters to dialogue and lyrics. The case study on Dou E Yuan reports higher expert ratings than a single-agent baseline, while ablations associate the visual agent with visual coherence and the speech agent with speech accuracy [130].

Cognitive-structured multimodal interaction.

The Cognitive-structured Multimodal Agent addresses long-horizon sessions that combine visual understanding, image generation, editing, composition, topic switching, and references to earlier images. A Perceptual Abstraction Engine stores image tags, descriptions, and thumbnails in episodic visual memory; a Cognitive Retrieval Engine selects relevant episodes; and a Multimodal Executive Controller dispatches the retrieved context to generation, editing, composition, or response modules. The reported experiments show improved visual retrieval and generation quality in 20-turn sessions through selective visual-context recall [131].

Game-development evaluation.

GameDevBench evaluates coding agents on Godot tasks that combine repository-scale code with sprites, shaders, animations, scene graphs, UI layouts, physics, cameras, and gameplay behavior. Its tasks provide reference solutions and deterministic tests, while optional editor screenshots and runtime videos expose visual and temporal state to evaluated coding agents. The benchmark shows that multimodal observation improves pass@1 on some configurations, while recurring failures involve node hierarchy, materials, layout, animation state, camera framing, and asset selection [136].

Node-Based Editing.

Node-Based Editing represents text, audio, images, and video as editable nodes in a multimodal graph. A task-selection agent routes requests among story generation, node-structure reasoning, diagram formatting, context generation, and node editing. Users can edit a node, branch a story, or regenerate a selected node while retaining the rest of the graph. The node graph therefore provides an intermediate representation for passing context between modalities and for limiting regeneration to the affected part of a multimodal artifact [132]. Figure 37 illustrates how this graph representation bounds regeneration to the edited node and its attached media.

Action Agent.

Action Agent separates language-guided navigation into trajectory imagination and low-level execution. In the first stage, an orchestration agent selects a video-generation pipeline, revises prompts through structured validation, stores successful strategies across tasks, and accepts a first-person navigation video when prompt adherence, physical plausibility, and visual quality meet the reported criteria. FlowDiT then maps the validated video and language instruction to continuous velocity commands for different robot embodiments. The paper reports 86% success for generated navigation videos across 50 tasks, 73.2% navigation success in simulation, and 64.7% task completion on a real Unitree G1 under open-loop execution [133].

Agentic AI for Personalized Physiotherapy.

Agentic AI for Personalized Physiotherapy organizes a rehabilitation pipeline around four cooperating agents. The Clinical Extraction Agent converts medical notes into kinematic constraints; the Video Synthesis Agent uses those constraints to create a patient-specific exercise demonstration; the Vision Processing Agent estimates body pose from camera frames; and the Diagnostic Feedback Agent compares the estimated motion with the constraints and issues corrective instructions. A shared patient state carries the prescription, extracted limits, pose, and feedback through the loop. The paper presents a prototype architecture, implementation description, and preliminary component estimates, so its contribution is a design feasibility result for a safety-oriented generative-video workflow [364].

Sol Video Inference Engine.

Sol Video Inference Engine applies agentic coordination to the inference stack surrounding video generation. For a specified model, hardware platform, and serving configuration, parallel skill agents optimize caching, sparse attention, token pruning, quantization, and kernel fusion. An integrator composes the candidate techniques into a full stack, and a human validator checks whether the resulting speedup preserves visual quality. Across Cosmos3-Super, LTX-2.3, and SANA-Video, the framework reports more than 2× end-to-end acceleration with near-lossless VBench quality [134].

PaperTalker.

PaperTalker generates academic presentation videos from LaTeX papers through a multi-agent pipeline: a Slide Builder refines Beamer layouts via a Tree Search Visual Choice module under VLM judgment, a Subtitle Builder extracts narration content, a Cursor Builder grounds pointer movements with UI-TARS and WhisperX, and a Talker Builder synthesizes personalized speech and talking-head animation, with slide-wise parallelization for efficiency [135]. The accompanying Paper2Video benchmark pairs 101 research papers with author-recorded presentation videos and evaluates meta similarity, arena-style preference, audience comprehension quizzes, and presenter identity retention.

11. Training Agentic Visual Generation

Training an Agentic Visual Generation system is not equivalent to improving a visual generator in isolation. The generator determines which artifacts are reachable. The agentic loop determines how goals are decomposed, tools are selected, evidence is inspected, failures are diagnosed, results are revised, and execution is stopped. The central training problem is therefore to improve the policy that maps an evolving visual state, execution history, and user interaction to the next action. Fine-tuning a diffusion model, code model, or editor changes the executor’s capabilities and the set of reachable artifacts. It constitutes agentic training only when the update also changes state-dependent decisions or the behavior of the loop.

This distinction separates three practices that recent studies often place under the same label. Direct policy training updates a controller, planner, router, critic, or integrated multimodal policy through trajectories, preferences, or reinforcement learning [21,22,25,29,35,38,241]. Agentic data curation uses agents to construct data for a conventional visual model [216,289,365]. Persistent non-parametric adaptation freezes model parameters but updates persistent memory, skills, or routing knowledge from accumulated experience [44,45,220,233,236]. All three contribute to capability acquisition, but they support different claims. Agentic data curation does not show that the curator learned to act. Likewise, retained experience does not make a training-free system parametrically trained.

The chapter therefore follows the causal path by which experience changes future behavior. Section 11.1 identifies the trainable objects and establishes the boundary of agentic training. Section 11.2 examines the construction of visual trajectories and the supervision they preserve. Section 11.3 studies supervised and preference-based policy initialization, and Section 11.4 turns to reinforcement learning from multimodal and executable feedback. Section 11.5 distinguishes parametric experience distillation from non-parametric memory and skill updates. Section 11.6 then considers joint optimization, transfer, forgetting, provenance, and the evidence required to substantiate cross-task improvement.

11.1. Training Targets and Boundaries

The first question is not which optimization algorithm is used, but which component or persistent knowledge store is changed. In AVG, updates may target the generator, controller, router, critic, memory, or an integrated policy. Each object governs a different part of system behavior and therefore requires distinct data, objectives, and evidence. Generator updates can improve execution quality without changing tool selection. Controller updates, by contrast, can reduce redundant actions without changing the generator. Verifier updates reshape the feedback or reward landscape, while memory and skill updates can alter later behavior without modifying model parameters. Table 3 summarizes these causal roles.

Updates to the controller, or to the decision-making component of an integrated policy, provide the most direct evidence of agentic training under the survey’s operational definition. GenAgent first establishes tool use through supervised fine-tuning, then optimizes multiround decisions with outcome and process rewards [21]. TIR-Agent learns when to inspect, which restoration tool to apply, and when to stop [241]. Generation Navigator formulates text-to-image generation as state-conditioned action selection [29]. CanvasAgent learns executable reasoning-action trajectories and adapts tool choices after observing intermediate canvas states [25]. These systems train the mapping from observed state to subsequent action rather than the visual backbone alone.

Generator training has a neighboring but distinct causal role. A stronger executor enlarges the set of reachable artifacts, but an unchanged controller may still choose poor tools or compose them badly. OPERA addresses this mismatch by optimizing a restoration planner and co-training tools under the plans in which they are composed [273]. This coupling provides stronger evidence of agentic training than independent tool fine-tuning. The Manim study similarly separates code-generation improvements from visually grounded rewards on rendered animations [366]. Executor training can contribute to the training of an AVG system when it is coupled to the agentic loop, as in plan-conditioned or jointly optimized execution. However, executor improvement alone is not evidence that the agentic policy has learned. Stronger evidence requires the update to change observation-conditioned decisions or to be jointly optimized with the controller under loop-level feedback. Ordinary visual-model pretraining therefore remains background.

Agentic data curation follows the same boundary. ScaleEdit-12M, DataEvolver, and JAVEDIT use agentic workflows to construct, filter, or verify visual training data [216,289,365]. These systems can improve supervision for a downstream generator, but they do not show that the data-producing agent acquires a reusable policy. Section 11.2 therefore treats them as data sources and as cases of provenance and filtering risk, not as direct evidence of controller learning.

Persistent non-parametric adaptation must be separated from parametric training and within-episode adaptation. OctoT2I updates routing knowledge without supervised fine-tuning [44]. COMFYCLAW converts trajectories and feedback into reusable skills [45], while restoration systems retain high-value experience for later tasks [220,233,236]. Such persistent updates support stronger autonomy claims only when they change behavior on held-out future tasks. A longer context, extra reflection, or disposable search procedure demonstrates local adaptation rather than cross-task learning. CRAFT, VISTA, and Qwen-Image-Agent are useful boundary cases because they emphasize training-free or test-time operation [17,64,215]. Section 11.5 specifies the evidence needed to distinguish episodic reuse, persistent adaptation, and continual self-improvement.

11.2. Visual Trajectory Data and Supervision

Direct training of agentic policies benefits from supervision that distinguishes decisions and their consequences rather than specifying desirable final artifacts alone. A prompt–image pair defines an acceptable endpoint but does not reveal the evidence behind tool selection, revision, or stopping. A visual trajectory can instead preserve the task, pre-action state, selected action and arguments, resulting observation, critique or reward, and updated plan or stopping decision. Preserving these dependencies makes it possible to learn how behavior should change as the artifact evolves, rather than merely imitate successful output sequences.

Successful executable trajectories provide a practical starting point because they preserve both action syntax and observable consequences. CanvasAgent learns multiround orchestration from fully annotated CanvasCraft trajectories before reinforcement learning [25]. TOOLCAD similarly grounds actions in a CAD environment with executable intermediate operations [83]. Success-only traces, however, introduce selection bias: they show what worked but provide limited evidence about how to recognize deterioration, recover from failed calls, or distinguish necessary from redundant actions. Failure–diagnosis–repair trajectories are therefore especially useful when the target behavior includes reflection, correction, rollback, or adaptive stopping.

Pairwise trajectory construction provides complementary supervision by exposing relative decision quality. GenAgent compares final images and consecutive rounds while filtering revisions that reduce quality [21]. GenEvolve contrasts tool-orchestrated trajectories for the same request and structures their differences as reusable experience [43]. Clarify Before Executing similarly selects high-scoring trajectories and derives preference pairs from them [85]. Such data make the consequences of alternative decisions more explicit, although their usefulness still depends on the reliability and independence of the evaluator used to construct the comparison.

Long visual trajectories also impose a systems constraint. Each rollout may contain large intermediate images, repeated generator calls, executable states, and multimodal critiques. InterleaveThinker therefore decomposes expensive interleaved trajectories into single-step planner and critic data with step-wise rewards [281]. VisionCreator instead uses a simulated environment to reduce repeated interaction with external tools [38]. Both strategies improve scalability, but they discard different information: single-step optimization weakens full-trajectory credit assignment, whereas simulation introduces environment mismatch. Reliable comparison therefore benefits from reporting trajectory length, generator calls, simulator assumptions, filtering rates, and the retained distribution of failures rather than only prompt or image counts.

Agentic data curation introduces a related but distinct source of supervision. ScaleEdit-12M, DataEvolver, and JAVEDIT use agents and task-aware verifiers to construct, filter, or validate visual training data [216,289,365]. Such pipelines can substantially expand downstream supervision, but retrieved or generated assets and model-based judgments may also introduce provenance, licensing, evaluator-circularity, and leakage risks. Evidence is therefore stronger when data sources, model and tool versions, filtering and deduplication procedures, and overlap between training and evaluation pipelines are made explicit. Otherwise, additional scale may amplify shared evaluator bias or benchmark-specific patterns rather than improve transferable decisions.

11.3. Supervised and Preference-Based Policy Initialization

Supervised fine-tuning (SFT) primarily establishes an executable initial policy. It can teach valid action syntax, plan structure, state-referenced tool arguments, observation-conditioned reasoning, and plausible stopping behavior. GenAgent uses tool-use and reflection supervision before reinforcement learning [21]. CanvasAgent first learns executable reasoning–action trajectories [25], while TIR-Agent uses SFT before optimizing exploration and efficiency [241]. Across these systems, supervised initialization stabilizes the interface between reasoning and visual execution, but does not by itself guarantee effective decisions outside the demonstration distribution.

The composition and ordering of supervised data can matter as much as its volume. VisionCreator argues that generation-only SFT may erode broader multimodal ability, whereas indiscriminate mixture training may underdevelop creation expertise. Its progressive specialization curriculum preserves general capability while preparing the policy for later reinforcement learning [38]. VisionCreator-R1 provides complementary supervision for planning and reflection before multitask reinforcement learning [39]. IEA combines expert-distilled SFT, group relative policy optimization, and synthetic fine-tuning [275]. Together, these systems show that supervised alignment may be organized as a staged curriculum and, in some cases, revisited after reinforcement learning rather than serving only as a one-time initialization stage.

Decomposition provides another way to make visual policy initialization tractable. InterleaveThinker trains planner and critic components separately using step-level supervision rather than full expensive interleaved trajectories [281]. DiTTo constructs order-aware restoration trajectories offline and trains a policy from them [274]. Such decomposition can simplify optimization and diagnosis, but it also introduces interface risks: an abstract planner may propose unrealizable actions, while a transition-level critic may overlook long-horizon interactions. Supervised competence therefore remains most meaningful when the resulting decisions are validated against executable or rendered consequences.

Preference optimization complements supervised imitation when several trajectories are plausible but their relative quality can be judged. The trajectory pairs described in Section 11.2 can supervise choices such as whether clarification is needed, how much planning is appropriate, or which repair should be preferred. Clarify Before Executing uses selected trajectories for SFT and preference pairs for direct preference optimization [85]. GenEvolve and GenAgent likewise use comparisons between candidate trajectories or consecutive visual states in later optimization or distillation stages [21,43]. Preference objectives capture trade-offs that token-level imitation cannot express directly, but remain sensitive to evaluator bias, generator stochasticity, and user preferences that are absent from the comparison data.

The main limitation of supervised and preference-based initialization is coverage. Demonstrations encode only the tools, failures, visual conditions, and stopping conventions encountered during data construction. An agent may reproduce polished trajectories yet fail when a new artifact, tool, or generator changes the transition dynamics. Interaction-based optimization addresses part of this limitation, but reinforcement learning does not simply replace supervised initialization. SFT establishes executable and structured behavior; preference learning introduces relative trade-offs; subsequent interaction-based learning can then optimize decisions whose value becomes visible only after execution.

11.4. Reinforcement Learning from Multimodal and Executable Feedback

Reinforcement learning is particularly relevant to AVG because the quality of an action often becomes visible only after execution. A plausible prompt rewrite may lose identity, restoration may remove noise while damaging texture, and syntactically valid CAD code may still corrupt geometry. Additional revisions can also reduce an already acceptable result. The learning signal therefore depends on the visual or executable transition produced by an action. Such feedback can be delayed, stochastic, heterogeneous, and vulnerable to evaluator exploitation. The central distinction is therefore the source and temporal granularity of the reward rather than the particular policy-optimization acronym.

Table 4. Reward sources and failure modes in training Agentic Visual Generation policies.

Reward source	What it supervises	Granularity	Main failure mode
Format and tool-call validity	Action syntax; required fields; tool selection	Action or step	Formal compliance without visual usefulness
Executability and deterministic checks	Code execution; rendering; CAD validity; rule satisfaction	Step or trajectory	Runnable but visually or semantically poor artifacts
Constraint and task rewards	Object, relation, preservation, localization, or task success	Step or outcome	Sparse specifications encourage shortcuts
Perceptual and artifact-quality rewards	Realism, fidelity, restoration, aesthetics, and identity	State, transition, or outcome	Evaluator bias; generator stochasticity; metric conflict
Process and comparative rewards	Whether reflection or repair improves a later state	Transition or trajectory pair	Local gains may not improve the final trajectory
Human or learned preference	Open-ended quality, usefulness, style, and interaction trade-offs	Outcome or trajectory pair	Overoptimization; population mismatch

Table 4. Reward sources and failure modes in training Agentic Visual Generation policies.

Reward source	What it supervises	Granularity	Main failure mode
Format and tool-call validity	Action syntax; required fields; tool selection	Action or step	Formal compliance without visual usefulness
Executability and deterministic checks	Code execution; rendering; CAD validity; rule satisfaction	Step or trajectory	Runnable but visually or semantically poor artifacts
Constraint and task rewards	Object, relation, preservation, localization, or task success	Step or outcome	Sparse specifications encourage shortcuts
Perceptual and artifact-quality rewards	Realism, fidelity, restoration, aesthetics, and identity	State, transition, or outcome	Evaluator bias; generator stochasticity; metric conflict
Process and comparative rewards	Whether reflection or repair improves a later state	Transition or trajectory pair	Local gains may not improve the final trajectory
Human or learned preference	Open-ended quality, usefulness, style, and interaction trade-offs	Outcome or trajectory pair	Overoptimization; population mismatch

Executable feedback is particularly informative when relevant correctness conditions can be checked mechanically. TOOLCAD uses step- and outcome-level signals from a CAD environment before online curriculum reinforcement learning [83]. The Manim study combines code-side and visual signals because successful rendering does not guarantee fidelity to the instruction [366]. Similar distinctions arise in web, 3D, and diagram generation. Deterministic checks can establish executability or exact constraint satisfaction, whereas perceptual rewards assess properties that become visible only in the rendered artifact. Treating these signals separately prevents formal validity from masking poor visual or semantic quality.

Outcome rewards align optimization with the final task but create long-horizon credit-assignment problems. GenAgent combines final-image pointwise rewards with pairwise process rewards comparing consecutive states [21]. CanvasAgent likewise uses outcome and process signals while adapting tool choices from intermediate assets [25]. ImageEdit-R1 combines format and semantic-consistency rewards to coordinate specialized editing agents [35]. Denser process supervision can identify useful intermediate decisions, but excessive emphasis on process may reward unnecessarily long traces, whereas outcome-only supervision cannot determine which action caused success or damage.

Generation Navigator exposes a specifically visual form of this problem. A terminal-only reward provides coarse credit even when visual quality peaks early, declines after a later revision, or remains unchanged across wasteful turns. Its PRE-GRPO objective separates discovering a high-quality state, retaining that quality, and reaching it efficiently [29]. This suggests that AVG objectives benefit from modeling quality dynamics rather than final quality alone. In iterative creation, improvement, regression, recovery, and no-op behavior can carry different implications even when two trajectories end with similar artifacts.

Perceptual rewards are also noisy because the controller does not fully determine the rendered result. Gen-Searcher finds image-only rewards unreliable when a stochastic generator cannot realize a useful reference, motivating complementary text and image rewards [22]. TIR-Agent adapts the contribution of heterogeneous image-quality signals [241]. Restore-R1 uses multimodal perceptual feedback [271], while PaAgent balances perceptual, objective, and identity-sensitive criteria [288]. Reward composition is therefore most informative when its causal scope matches the component being optimized. Pure outcome rewards may penalize a good decision for an unlucky generator sample, while rewards detached from rendering may reinforce plans that are semantically plausible but visually ineffective.

The verifier or reward model can itself become part of the training problem. JarvisEvo jointly optimizes an editor and a human-calibrated evaluator [259]. Its ablations show that increasing internal scores can coexist with weaker external performance when evaluator calibration is inadequate. Updating the evaluator may help maintain a useful learning signal as the policy changes, but it also creates co-adaptation and removes evaluator independence. Claims based on such training are therefore stronger when the learned reward is calibrated against evidence outside the coupled training loop, such as held-out metrics, alternative evaluator families, or human judgments.

Finally, visual reinforcement learning is constrained by rollout cost. Repeated generation, rendering, execution, and multi-agent interaction can dominate training time, motivating offline trajectories, simulation, reward-model proxies, and component-wise optimization. InterleaveThinker separates expensive trajectories into planner and critic stages [281], while VisionCreator performs reinforcement learning in a simulated environment [38]. These approximations exchange environmental fidelity for scale and can introduce simulator or reward-model mismatch. Their training value therefore depends on whether policies learned from predicted or proxy consequences transfer back to the deployed generator, tools, and environment.

11.5. Experience Reuse, Skill Abstraction, and Distillation

Cross-task improvement is not restricted to gradient-based updates. An AVG system may retain episodes, abstract procedures, update routing knowledge, revise a verifier, or distill successful behavior into model parameters. These mechanisms differ in persistence, scope, and evidential strength. Passive records do not change behavior by themselves. Retrieval can support later decisions without changing model parameters, skills can compress recurring procedures across tasks, and parametric distillation can internalize experience but introduce forgetting. Table 5 organizes these forms of persistent cross-task adaptation.

Episodic retrieval provides a basic form of persistent cross-task adaptation without changing the base model. SIDiffAgent stores prior diffusion experience while keeping the underlying model training-free [233]. Self-Evolving Agentic Image Restoration indexes high-reward experience by degradation type [220]. EvoIR-Agent uses a hierarchical experience pool to guide later tool selection and ordering [236]. Such systems can reduce repeated search, but improved performance does not by itself establish learning in the parametric sense. Retrieval-disabled controls and held-out future tasks help distinguish genuine behavioral reuse from simply providing additional context or selected demonstrations.

Skill abstraction increases the scope of reuse by converting multiple traces into reusable procedures. COMFYCLAW distills workflow trajectories, execution errors, and verifier feedback into reusable Agent Skills across agents, backbones, and task splits [45]. A skill differs from episodic retrieval because it represents a procedure intended to instantiate in new contexts rather than a particular prior episode. This abstraction also broadens the impact of erroneous updates: a flawed skill can affect many later tasks. Evidence for skill-level adaptation is therefore stronger when transfer extends beyond the examples used to construct the skill and when the system preserves provenance and supports versioning, conflict handling, or rollback.

Router and policy-memory updates lie between episodic retrieval and parametric policy learning. OctoT2I uses self-interaction feedback to update the router’s external tool knowledge without supervised fine-tuning [44]. This non-parametric update can alter later generator or tool choices while leaving the router and generator parameters unchanged. Its causal effect is clearest when held-out decisions improve under fixed tools, generation budgets, and retrieval costs. Otherwise, apparent gains may arise from access to additional candidates or more expensive resources rather than better routing knowledge.

GenEvolve provides a contrasting example of parametric visual-experience distillation. It compares tool-orchestrated trajectories, uses a privileged teacher to structure their differences, and updates the agent through on-policy self-distillation and reinforcement learning [43]. Instead of retaining expensive search traces as context, the pipeline attempts to internalize their useful behavior into a reusable policy. Evidence for such distillation requires more than parameter change: post-update performance should improve independently of the experience-construction process, and comparison with direct retrieval is needed to determine whether useful decision knowledge has actually been internalized.

The survey’s L5 criterion requires more than persistent storage or within-episode correction: retained experience must change later decisions and transfer to held-out tasks. Section 11.6 examines the transfer, forgetting, budget, and governance controls needed to substantiate this claim.

11.6. Joint Optimization, Transfer, and Evidence Gaps

AVG systems distribute responsibility across interdependent modules. A planner trained against one executor may exploit its specific strengths and weaknesses, then fail when the generator changes. Independently trained tools may also perform well alone but interact poorly in sequence. OPERA jointly optimizes restoration planning and execution through reinforcement learning and agent-guided tool co-training [273]. This design enlarges the reachable behavior space but complicates attribution. Evaluation should separate planner-only, executor-only, and joint updates, using frozen-component and component-swap controls where possible.

Editor-evaluator coupling creates another form of co-adaptation. JarvisEvo updates both policies so the evaluator remains informative as the editor changes [259]. This may resist exploitation of a static reward model, but it creates a moving objective and removes evaluator independence. The critic may improve calibration, drift toward editor-specific patterns, or leak reward regularities into the policy. Final evidence should therefore come from held-out metrics, alternative evaluator families, or blinded human assessment. Integrated planner-generator-critic models likewise require role-specific ablations and independent evaluation despite lower interface overhead.

Cross-task transfer remains the principal unresolved question. Most studies evaluate with the same tools, generator family, evaluator family, or task distribution used to construct training trajectories. These experiments demonstrate in-distribution optimization, not reusable agency. Stronger protocols should separate unseen task instances, task families, tool combinations or versions, and generators or executors. They should also distinguish syntactic transfer from decision-quality transfer. An agent may call a new API correctly yet choose it at the wrong time or misread its visual failure modes.

Controlled deployment conditions are also required to substantiate training gains. Comparisons between pre- and post-training policies should match rollouts, generations, retrieved examples, tool calls, and revision turns. Otherwise, a larger test-time budget may appear as a training gain. Efficiency claims should compare both quality at matched cost and cost at matched quality. TIR-Agent and Generation Navigator treat redundant actions and trajectory efficiency as learned behavior [29,241]. These inference quantities serve as causal controls, not as an inference taxonomy.

Continual training exposes failures absent from one-shot fine-tuning. New experience may overwrite prior capabilities, faulty verifiers may promote harmful skills, and growing memories may retrieve stale or conflicting procedures. Evidence should include learning curves, retention on prior tasks, update provenance, conflict handling, and rollback to a known version. User or retrieved assets may also become embedded in shared memory, skills, or parameters, increasing privacy and copyright risk. Bounded update authority is therefore part of training design.

12. Evaluating Agentic Visual Generation

High artifact quality does not, by itself, demonstrate effective agency. A stronger evaluation must identify whether an agent satisfies the intended goal, makes competent intermediate decisions, improves the result because of feedback, and does so at an acceptable cost. This distinction follows the operational definition developed earlier in the survey: agency is evidenced when an observation of the evolving artifact, environment, or interaction history changes a later action. A visually appealing result may instead arise from a stronger base generator, repeated sampling, a larger inference budget, or chance. Evaluation must therefore separate the quality of the produced artifact from the behavior of the loop that produced it.

We organize the evidence into four complementary levels. Artifact-level evaluation asks whether the final visual result is perceptually and presentationally acceptable. Goal-level evaluation asks whether the requested content and constraints are satisfied. Trajectory-level evaluation examines whether planning, observation, diagnosis, repair, and stopping are competent and causally useful. System and human-centered evaluation asks whether these gains remain worthwhile under realistic resource, reliability, safety, and interaction constraints. Section 12.5 then identifies the main gaps that prevent these four levels from forming a unified evaluation protocol and states the minimum controls needed for reliable comparison.

12.1. Artifact Quality

Artifact-level metrics assess the intrinsic quality of the rendered result, including realism, aesthetics, perceptual fidelity, clarity, temporal stability, geometric appearance, and human preference. They remain necessary because an agentic trajectory that produces an inferior artifact has limited practical value. At the same time, artifact quality is not a modality-independent quantity. A sharp and aesthetically convincing image, a temporally coherent video, a cleanly rendered 3D object, a legible scientific figure, a well-composed slide deck, and a visually faithful interface expose different perceptual failures and therefore require different evidence. Recent evaluation frameworks accordingly combine modality-specific signals rather than treating a single image-text similarity score as a universal measure [47,48,87,104,128,225].

The meaning of artifact quality changes with the visual artifact and its mode of use. Structured-document evaluations, for example, combine rendered-page inspection, layout rules, readability checks, and human review to assess typography, alignment, overflow, and cross-page consistency [108,117]. Section 12.2 then turns from intrinsic artifact quality to goal and constraint satisfaction, while Section 12.3 and Section 12.4 examine the trajectory, system, and human conditions under which both are achieved.

No single evaluation protocol is sufficient across these domains. Reference-based metrics are useful when a target image, rendering, or layout is available. However, they can penalize valid alternatives in open-ended creation and may fail to isolate the region affected by an edit. Learned multimodal judges are more flexible, yet their scores depend on prompt formulation, perceptual sensitivity, and alignment with human judgments.

Tool-augmented evaluators address part of this limitation by grounding judgments in regions, image differences, scene descriptions, code, or other observable evidence. CIGEval illustrates this strategy by combining a multimodal evaluator with specialized visual tools across conditional image-generation tasks [47]. EdiVal-Agent instead separates global visual quality from instruction following and the preservation of unchanged content [48].

Human validation complements automated diagnostics in long-form video and procedural 3D benchmarks because aggregate perceptual scores do not fully capture multi-shot quality or the acceptability of a rendered shape [87,225]. These studies support an evaluator portfolio in which reference-based, learned, tool-grounded, and human signals play complementary roles.

The main limitation of artifact-level evaluation is that it describes the result without establishing why it was obtained or whether it is correct for the task. Image-centric metrics can reward outputs that are visually plausible while violating explicit structure, data, geometry, or functionality. SciFlow-Bench shows this failure directly: a scientific diagram can appear coherent while omitting components, reversing relations, or hallucinating labels, which motivates inverse parsing and structural recoverability measures beyond visual similarity [103]. In Web development, VISTA finds that visual fidelity and functional correctness are only partially coupled, so a screenshot that resembles the specification does not guarantee correct interaction behavior [128]. Likewise, 3DCodeBench reports that successfully rendered objects may still contain disconnected or floating geometric components [87]. Artifact quality is therefore a necessary first layer, but it cannot establish goal satisfaction, competent decision making, or effective use of feedback. The next level must evaluate the requested goals and constraints directly.

12.2. Goal and Constraint Satisfaction

Goal-level evaluation asks whether the produced artifact satisfies the operational specification rather than merely appearing plausible. A visual request may combine positive semantic requirements, including objects, attributes, relations, identities, and events, with prohibitions, geometric or structural constraints, factual and data requirements, soft preferences, preservation requirements, and executable behaviors. These requirements are heterogeneous: the presence of an object, the direction of an arrow, the dimensions of a room, the value encoded by a chart, and the behavior of a button are not instances of the same measurement problem. Goal satisfaction should therefore be represented as a set of verifiable conditions rather than compressed prematurely into a single perceptual score [6,48,49,103,104,128].

The verifier must consequently match the type and granularity of each constraint. Object and attribute requirements can be checked with detectors, localized crops, and multimodal semantic judgments. Spatial and relational constraints require symbolic or geometric evidence. Blueprint-Bench, for example, compares room-connectivity graphs and relative size rankings rather than judging floor-plan appearance alone [6]. SciFlow-Bench inverse-parses rendered scientific diagrams and compares recovered nodes and edges with canonical graphs, thereby testing whether the intended structure survives rendering [103]. Executable artifacts require visual judgments to be combined with DOM-grounded matching, browser tests, or workflow-based interaction tests, as in VISTA and Vision2Web [128,129]. Physical consistency can be tested through forward-system simulation, as in ImagingBench [49]. These examples show why a preference model cannot substitute for exact geometry, graph structure, executable behavior, or physical validity.

Editing tasks add an asymmetric requirement: the requested region or relation must change, while unrelated content must remain stable. EdiVal-Agent makes this distinction explicit by separating instruction following, content consistency, and global visual quality. Its evaluator combines detector-based symbolic checks, semantic judgments on detector-guided crops, similarity measurements for unchanged objects and backgrounds, and a human-preference model for overall visual quality [48]. HOI-Edit further shows that relation editing cannot be reduced to isolated entity detection. A successful human-object interaction edit must establish the requested interaction while preserving the grounded human-object pair, which motivates region-sensitive evaluation of both relational plausibility and entity retention [69]. In multi-turn settings, these conditions should be checked after each revision against an evolving specification because an earlier satisfied constraint may be lost during a later edit.

Reporting must preserve this decomposition. A defensible protocol should provide per-category success rates, an all-constraints-satisfied rate, and separate preservation scores where editing is involved. Hard constraints should act as validity conditions: a missing component, reversed dependency, incorrect data value, or broken interaction should not be compensated for by stronger aesthetics elsewhere. Soft preferences and human judgments remain appropriate for properties such as communicative clarity, style, or pedagogical effectiveness, but they should be reported separately from deterministic checks [95,104]. This separation also makes failure diagnosis more informative. An aggregate score may show that a system underperforms, whereas category-level results reveal whether the bottleneck is semantic grounding, spatial reasoning, structural recoverability, factual correctness, or executable behavior [48,103,128].

Goal-level success nevertheless remains an outcome property. It does not establish that the agent selected competent intermediate actions or that feedback caused the successful result. A fixed pipeline, a stronger generator, or repeated candidate search may also produce a constraint-compliant artifact. Evaluation must therefore proceed from whether the goal was met to how it was met. The next level examines planning, tool selection, diagnosis, revision, recovery, and stopping as decisions along the generation trajectory, using matched alternatives to test whether adaptation is responsible for the gain.

12.3. Trajectory and Decision Quality

Trajectory-level evaluation asks two progressively stronger questions: whether the agent makes competent decisions along the generation trajectory, and whether its adaptation to intermediate evidence causally improves the result. Answering either question requires more than a chronological list of prompts, reasoning text, or tool calls. An evaluable trace should expose the task state available before each decision, the selected action, the resulting visual or executable observation, the feedback derived from that observation, and the state, plan, or action updated next. The basic unit of analysis is therefore an observable state–action transition. The presence of a component named planner, critic, or memory does not establish that the component used evidence correctly or contributed to the outcome [70,128,362].

Decision correctness can then be decomposed according to where a choice is made in the loop. Planning translates the goal into executable and testable dependencies; action selection chooses a necessary tool with valid parameters; diagnosis identifies the actual failure at an appropriate location and granularity; revision or recovery repairs that failure without damaging already satisfied constraints; and stopping terminates the process when further actions no longer provide sufficient expected value. Table 6 pairs the observable failure modes at each decision point with the intervention needed to test them.

Existing benchmarks provide partial evidence for decision correctness, but they also show why trace observability must not be confused with competence. ATP-Bench evaluates tool planning through tool-call precision, missed opportunities, parameter checks, placement checks, and comparison with curated tool sets. By separating planning from changing tool backends, it offers a relatively clean diagnosis of model-side selection behavior [118]. Its boundary is equally important: a correct tool plan does not show that executing the plan improves the visual artifact. VideoWeaver moves closer to end-to-end analysis by reading execution traces, tool-call records, metadata, intermediate files, and final videos, and by reporting both process and output metrics [70]. However, several of its process scores indicate whether a checkpoint occurred rather than whether the associated decision was correct. These benchmarks therefore support a distinction between trace coverage, decision correctness, and downstream effect.

Matched counterfactuals are needed to establish the causal utility of adaptation by intervening on the loop rather than observing it alone. The most informative comparison holds the input, executor, tool access, and resource budget constant, and then moves from single-pass generation to repeated sampling or best-of-N selection, a uniform retry policy, a matched fixed workflow, and finally an adaptive agent. A no-feedback condition tests whether observations are necessary, while corrupted-feedback and oracle-feedback variants test robustness to misleading evidence and the upper bound supplied by correct diagnosis. Each control removes a different alternative explanation: more samples, deterministic repair, a stronger preset workflow, or privileged feedback may otherwise be mistaken for learned decision making.

Available evidence illustrates the value of this counterfactual design. GameDevBench varies whether otherwise comparable coding agents receive editor screenshots, runtime video, both, or neither. Visual feedback raises GPT-5.4 pass@1 from 41.1% to 52.0% in the reported setting, but the gain varies across model–harness combinations and some configurations decline [136]. The result supports a causal benefit for feedback within matched configurations rather than a universal benefit from adding more feedback. 3DCodeBench compares single-turn generation, a uniform stateless retry loop driven by Blender error traces, and autonomous coding-agent harnesses under fixed time budgets. Retry and agentic execution both improve executability, but the full harness does not improve conditional shape quality among scripts that already execute [87]. Blueprint-Bench likewise finds no meaningful gain from agent-based iterative refinement over single-pass spatial reconstruction [6], while ImagingBench reports small and inconsistent changes when an adaptive Planner replaces a strong fixed Expert prompt and notes that its protocol does not establish causal use of the forward model [49]. Together, these results show that an adaptive agent should be compared with strong non-adaptive alternatives rather than with single-pass generation alone.

Recovery, rollback, and stopping reveal where an adaptive trajectory fails. A useful protocol should distinguish four failure sources. Diagnosis bottlenecks occur when the system does not identify the relevant error. Policy bottlenecks arise when the error is recognized but the next action is inappropriate. Executor bottlenecks occur when a sound action fails to produce the intended visual change. Stopping bottlenecks arise when the system continues after reaching its best available state. Corresponding measures include error-localization accuracy, repair success, collateral-damage rate, repeated-failure rate, rollback success, and post-peak quality degradation. Controlled execution errors and corrupted feedback can test whether the agent recovers from adverse evidence rather than merely following any critique it receives. These tests remain proposed protocol elements rather than a mature shared standard, because current AVG benchmarks expose only subsets of the required failure–repair cycles [70,87,128].

Trajectory quality must finally be normalized by the opportunity and cost of adaptation. The shortest trajectory is not necessarily the best, because verification and rollback add useful steps, while a longer trajectory may simply repeat failed actions. Evaluation should therefore report calls to the first valid artifact, unnecessary-action rate, successful repair per call, improvement per generator invocation, and quality–cost curves, together with outcome quality under common call, time, and compute limits. VISTA demonstrates why behavioral descriptors cannot substitute for these competence measures. Its Surgical Diff Score characterizes localized patching versus full-file rewriting, but is only weakly related to task success and partly depends on the tools exposed by the harness [128]. Editing locality and raw path length describe how a system acts; they do not independently establish that it acts well.

The unit of reporting is consequently the complete model–harness–tool–feedback–budget configuration rather than the language model alone [102,128]. Trajectory studies should disclose model and harness versions, available tools and skills, feedback sources, call and time limits, retry and failure-handling rules, and variation across repeated trials. Trajectory-level evaluation should separate three claims.

Exposed reasoning, repeated self-critique, or additional calls establish none of these claims on their own. Even a causally useful trajectory may remain too expensive, variable, unsafe, or difficult for users to inspect and override; Section 12.4 therefore evaluates whether trajectory gains remain worthwhile at the level of the deployed system and its surrounding human interaction.

12.4. System and Human-Centered Evaluation

System-level evaluation asks whether gains attributed to an agentic loop remain worthwhile once resources, operational failures, and human labor are made visible. The evaluated unit is the deployed model–harness–tool configuration, including the generators and evaluators used, tool interfaces, execution environment, recovery policy, and autonomy setting. A system that obtains a strong artifact through costly calls, unstable retries, or extensive human correction is not equivalent to one that reaches comparable quality reliably and autonomously. These measures should therefore accompany artifact-, goal-, and trajectory-level results [70,102,128,362].

Table 7. System and human-centered dimensions for evaluating deployed Agentic Visual Generation.

Dimension	Recommended observations and measures
Resource demand	Model and tool calls; tokens; GPU time; latency; monetary cost; time to first valid artifact; quality–cost frontier
Reliability and repeatability	Mean and dispersion across runs; confidence intervals; pass@k; passk; timeout, crash, and refusal rates; repeated failures; recovery rate
Operational reproducibility	Model, harness, tool, prompt, evaluator, and dependency versions; seeds; budgets; retry rules; logs; intermediate artifacts
Safety and privacy	Unsafe artifacts and tool actions; policy bypasses; red-team attack success; sensitive-data exposure, transmission, retention, and provenance
Controllability and human authority	Autonomy mode; approvals and rejections; intervention opportunities; override and rollback success; state visibility; autonomous completion rate
Workload, accessibility, and trust calibration	Completion time; turns and corrections; workload and usability scales; accessibility barriers; error detection; appropriate reliance; expertise-stratified results

Table 7. System and human-centered dimensions for evaluating deployed Agentic Visual Generation.

Dimension	Recommended observations and measures
Resource demand	Model and tool calls; tokens; GPU time; latency; monetary cost; time to first valid artifact; quality–cost frontier
Reliability and repeatability	Mean and dispersion across runs; confidence intervals; pass@k; passk; timeout, crash, and refusal rates; repeated failures; recovery rate
Operational reproducibility	Model, harness, tool, prompt, evaluator, and dependency versions; seeds; budgets; retry rules; logs; intermediate artifacts
Safety and privacy	Unsafe artifacts and tool actions; policy bypasses; red-team attack success; sensitive-data exposure, transmission, retention, and provenance
Controllability and human authority	Autonomy mode; approvals and rejections; intervention opportunities; override and rollback success; state visibility; autonomous completion rate
Workload, accessibility, and trust calibration	Completion time; turns and corrections; workload and usability scales; accessibility barriers; error detection; appropriate reliance; expertise-stratified results

Resource accounting reveals trade-offs hidden by endpoint rankings. In GameDevBench, multimodal feedback increases monetary cost in most settings, reaching 3.3 times in one GPT-5.4 configuration, while gains vary by model and harness [136]. 3DCodeBench jointly reports per-query cost, output tokens, wall-clock time, throughput, and human-preference Elo, revealing a saturating cost–quality relationship [87]. VideoWeaver records tool calls, language, image, and video tokens, and generation time, showing that stronger long-video workflows often consume more planning and coordination resources [70]. SciVisAgentSkills further finds that identical skills reduce token use in Claude Code but increase it in Codex [102]. Comparisons should therefore use common call, time, and monetary budgets and report Pareto frontiers or quality at fixed budgets.

Reliability differs from obtaining one favorable sample. A best run or pass@k rewards additional attempts, whereas pass^k and failure distributions measure consistent success. SciVisAgentSkills runs each configuration three times and reports the mean, standard deviation, pass@k, and pass^k [102]; this is a useful minimum but provides limited precision for heterogeneous, long-horizon tasks. Stronger protocols should publish per-case results and seeds, distinguish model stochasticity from benchmark sampling and human-rating uncertainty, and record timeouts, crashes, refusals, recovery attempts, and cost variance. Model, harness, tool, prompt, dependency, and evaluator versions should also be fixed or disclosed, because performance depends on the surrounding configuration [70].

Human-centered evaluation asks whether initiative is allocated effectively and accountably. HiLSVA conducts a controlled study with twelve participants of varied expertise across full-auto, half-auto, and mixed-initiative settings [100]. Mean execution time rises from 9.83 to 11.67 and 13.50 minutes, but the overall difference is not significant; the result suggests a possible oversight–efficiency trade-off rather than a confirmed speed effect. Participants value action approval, autonomy adjustment, feedback, provenance, and the ability to revisit earlier steps. However, the questionnaire measures perceived transparency rather than calibrated trust, and the design does not isolate adaptation from the additional human feedback it receives. Evaluations should therefore log human-provided information, corrections, approvals, and direct manipulations, and stratify results by autonomy mode and expertise [100].

Personalization is not a substitute for interactive evaluation. DirectorBench uses seven synthetic user profiles and fourteen annotators to expose preference-specific weaknesses in long-form video, showing why equal-weight averages can hide failures for particular users [225]. It does not, however, measure real intervention, workload, or trust calibration. Interactive studies should test whether users detect errors, intervene at consequential moments, override or roll back actions, and rely appropriately on the system. Accessibility should likewise be tested—not inferred from a natural-language interface—across input methods, perceptual and motor demands, language, expertise, and alternative representations.

Safety belongs to the full loop, not only the final frame. RedEdit shows that an adaptive editing agent can make 76.2% of unsafe images evade a black-box classifier in fewer than two edits on average while retaining 93.0% of their malicious semantics [46]. Although designed for red-teaming, the study demonstrates how repeated observation and revision can expand the attack surface beyond endpoint moderation. Evaluation should inspect unsafe intermediate artifacts, tool actions, external transmissions, policy bypasses, and the cumulative effect of individually benign edits. Privacy reporting should likewise track which user assets enter APIs, evaluators, logs, caches, or shared memory, who can retrieve them, and for how long. Because such measurements remain rare, missing privacy evidence should be treated as an evaluation gap rather than evidence of safety.

No single score can combine quality, cost, reliability, safety, and human authority without hiding value judgments. A more informative comparison reports a quality–resource–reliability profile under a declared autonomy setting, supplemented by safety and human-centered results. This profile can identify Pareto-dominated systems while keeping normative trade-offs visible. Section 12.5 summarizes the gaps that still prevent such reporting from becoming a shared evaluation protocol.

12.5. Remaining Evaluation Gaps

The central gap is not a lack of endpoint metrics, but the fragmentation of evaluation across artifacts, goals, trajectories, and deployed systems. Most benchmarks specialize in one modality or one evaluation level and do not connect final quality and constraint satisfaction with the decisions, failures, resource use, and human interventions that produced them. Consequently, a high endpoint score may show that a system works in a particular configuration without establishing whether the agentic loop caused the gain or whether the gain remains worthwhile at matched cost [48,70,87,128].

Trajectory and feedback evidence is particularly limited. Few benchmarks release action-level traces or compare adaptive agents under equal budgets with fixed workflows, uniform retry, no-feedback, and corrupted-feedback conditions. Evaluator circularity creates a related gap when the training reward, online critic, stopping signal, and final judge share models or prompts [87,136,259]. Minimum controls for reliable comparison include a task-validated external evaluator or blinded human assessment, repeated trials, matched resource budgets, and disclosure of the model–harness–tool configuration and scoring protocol [47,70,104].

13. Challenges and Research Frontiers

Agentic visual generation is developing along three broad paths: expanding the scope of visual agency, increasing the intelligence available to visual agents, and establishing reliable evaluation. Scope expansion concerns the artifacts, project durations, tools, environments, application consequences, and related operating conditions that a system can handle. Intelligence improvement concerns foundation models, system architectures, reasoning, process training, continual learning, collaboration, deployment, and related capabilities. Evaluation concerns the quality, behavior, cost, and consequences of the complete creation process. The three paths also interact: longer projects tend to expose control failures that short tasks leave hidden, a stronger controller magnifies the cost of weak generation and weak verification, and claims about planning, memory, or self-improvement call for evidence at the process level. The subsections below organize each path around concrete failure modes and the research directions they motivate.

13.1. How to Expand the Scope of Visual Agency

13.1.1. Multimodal Creative Workflows

A multimodal creative workflow produces several related artifacts during one project. Text, images, audio, video, source code, layouts, data, and interactive states all contribute to the final work, and each artifact carries its own representation, editing interface, and failure surface. Producing each artifact is within reach of current systems; a harder problem lies in preserving the relations among them. A character that appears in a storyboard, a narration script, a generated video, and a subtitle track is one identity distributed across four representations, and a change to any one of them propagates to the others through constraints that current systems do not represent explicitly.

Existing systems handle each modality through a separate operation with a limited handoff. The dominant pattern is serial conditioning: a text plan drives an image model, whose output conditions a video model, whose frames are passed to a voice or editor stage. Each handoff re-encodes the artifact into a form that loses part of the upstream structure, and a revision in an early stage forces regeneration of everything downstream. The technical causes are threefold. Identity is re-derived at every generation from a reference image or a textual description, and this is a main reason long-form storytelling still exhibits character drift: facial features, wardrobe, and even body proportions subtly change between scenes because the generative process has no persistent binding to a subject. Constraints such as “the same room”, “the same brand palette”, or “the same data underlying the chart” are expressed only in the prompt and therefore decay through the chain. The dependency structure among artifacts is implicit in the sequence of calls, so the system has no way to determine which outputs must be refreshed when one input changes.

A useful workflow abstraction treats a project as a graph of linked artifacts with typed dependencies, shared references, and version-stable identities. Under this view, the interesting research problems become concrete: how to factorize a subject into a representation that survives re-rendering across styles, camera angles, and modalities; how to express cross-modal constraints so that a verifier can check them at each step; how to localize an update so that regeneration touches only the artifacts whose dependencies changed, in the way a build system invalidates targets; and how to preview the propagation of a proposed edit before executing it. Progress on these problems would turn a multimodal workflow from a chain of independent calls into an editable artifact in its own right.

13.1.2. Long-Horizon Creative Projects

A long-horizon project contains many decisions distributed across turns, scenes, files, and production stages. Later actions depend on earlier requirements, intermediate results, unresolved issues, and user preferences, and the system must carry these forward while the context grows. Short-task success gives limited information about this ability: recent empirical studies of long-horizon agents report a consistent pattern in which per-step success that looks strong on isolated operations drops sharply once tasks require an extended, interdependent action sequence, and the dominant failure categories shift from execution errors toward planning errors, loss of early constraints, and error accumulation through history.

Creative generation inherits these failure modes and adds visual ones. Identity drift, layout drift, obsolete references, and repeated repairs are the visual counterparts of forgetting a constraint or acting on a stale assumption. A system may preserve the conversation while losing the exact artifact version that a decision referred to; it may continue executing a plan after the user has revised a central requirement, because the constraint is still in the context window and no longer attended to during generation. The underlying mechanisms are understood: attention over a long trajectory dilutes early instructions, compressed memory keeps the language while dropping the visual state it referred to, and project-level state remains conflated with dialogue history.

Closing this gap requires project-level state that is independent of the conversation. Such state would connect requirements, artifacts, versions, decisions, and unresolved issues in an explicit record; it would separate stable commitments from provisional choices; and it would expose the dependencies that a proposed revision will touch before it executes. Checkpointing, branching, and rollback—the standard machinery of versioned creative tools—have direct analogues in agent memory, but they need to be lifted from file-level snapshots to decision-level provenance: which observation justified this edit, which requirement it served, and which later decisions depend on it. Equally important is an explicit notion of commitment hygiene: detecting when a later instruction contradicts an earlier one, asking whether the earlier commitment should be revised, and propagating that revision across the affected decisions. Long-form video, multi-page documents, interactive stories, and complex image editing all become tractable in proportion to how much of this bookkeeping the system can carry reliably.

13.1.3. Open Tool Environments

An open tool environment contains changing models, applications, code runtimes, browsers, simulators, and data services whose interfaces, costs, permissions, and output formats vary across tasks and over time. Many visual agents are built against a fixed tool library with known interfaces, and the distance between that assumption and the deployed world now limits several domains: GUI agents that perform well on the applications seen during training drop sharply on unseen sites and cross-application workflows, because grounding, action semantics, and layout priors transfer poorly; a version bump in an image model can change aspect-ratio behavior, safety filters, or default styles in ways that silently break downstream stages; and a tool that returns a valid file in the wrong representation forces an extra conversion step that the planner did not budget for.

Three capabilities separate an open-environment agent from a closed-library one. The first is interface acquisition: maintaining capability descriptions for tools, learning their argument and return conventions from documentation and execution feedback, and treating a tool’s first use in a project as a probe. The second is cost-aware selection: choosing among tools that satisfy a request by expected quality, latency, financial cost, side effects, and the representation that the next stage requires. The third is graceful degradation: when a preferred tool is unavailable or misbehaves, re-routing to a substitute, falling back to a more conservative operation, or surfacing the failure to the user with an explanation. Each capability has a direct research formulation—tool documentation as a learning signal, tool selection as decision making under uncertainty, and tool failure as a first-class state in the controller—and they remain largely absent from current systems, which still assume a static action space.

Open environments also shift the trust model. Tools carry their own failure modes: a browser carries untrusted content, an IDE executes code, a simulator returns observations that a planner will act on, and an external model can return outputs that violate the project’s constraints. An agent that treats every tool output as ground truth inherits whatever the tool got wrong, including adversarial content embedded in retrieved pages or rendered screenshots. Treating tool outputs as evidence to be verified, with provenance and confidence attached, supports the move from closed demonstrations toward practical deployment.

13.1.4. Embodied and High-Consequence Domains

In an embodied or high-consequence domain, a generated artifact supports an action, a decision, a physical process, or a regulated activity. The artifact is evaluated through domain constraints and downstream consequences in addition to visual appearance, and the cost of a plausible-looking error scales with the stakes of the setting. Visual agents are entering navigation, physical simulation, engineering, scientific analysis, education, and public communication, and each domain imposes constraints that general-purpose models do not carry: a sim-to-real gap between what a generated scene predicts and how the physical world responds; a requirement that technical drawings, medical illustrations, and safety diagrams satisfy standards that appearance metrics leave unmeasured; and an asymmetry between reversible and irreversible actions that current controllers rarely represent explicitly.

A central technical bottleneck is the gap between generation and prediction. Photorealism and world-modeling are separate properties: empirical studies of physics in generated video show that current systems capture correlations from training data, generalize within the training distribution, and fail when the initial conditions leave it, a pattern consistent with case-based recall of training examples. Treating a generative model as a simulator requires properties that have to be established experimentally: causal, action-conditioned prediction; persistent state over an extended horizon; and physical accuracy on the events that matter for the downstream decision. These are also the properties that distinguish a world model from a video generator, and progress on them transfers directly to planning, since the same model can then be used to roll out candidate actions before committing to one.

High-consequence deployment also requires an explicit autonomy policy. The acceptable level of autonomy depends on the consequence of an error, and a system that applies a single trust threshold across actions treats a low-cost retry and an irreversible release alike. Risk-adjusted autonomy assigns different approval requirements to different classes of action, monitors domain constraints during execution, exposes calibrated uncertainty so that a human reviewer knows when to intervene, and preserves a complete record of the assumptions, references, and checks behind a result. These are control decisions that the architecture has to support natively, and they shape how far a visual agent can move past the demonstration regime.

13.2. How to Increase the Intelligence of Visual Agents

13.2.1. Generative and Multimodal Foundation Models

Foundation models provide the perceptual and generative capabilities that every higher-level agent decision depends on. Their failure modes are well documented and persist across scales: spatial relations, counting, in-image text, precise local edits, identity preservation under change, rare factual knowledge, and physically correct dynamics remain weak spots even in models that produce photorealistic single samples. A related structural weakness is the mismatch between understanding and generation. A model that interprets an image accurately may have no interface for making an exact change to the underlying representation, so an edit that a human would describe as a one-parameter adjustment becomes a re-generation with collateral change to preserved regions; a model with strong generation may in turn lack the introspection to know which region of its own output violates the instruction. These failures share a root cause: appearance quality is a training signal that does not force the model to internalize the structure that makes an edit local, an identity stable, or a scene physically consistent.

Architecturally, the field is converging on unified models that handle understanding and generation in one backbone, in three broad forms: pure autoregressive, pure diffusion, and hybrid designs that autoregress over semantic tokens while diffusing over pixels. The unification matters for agents because it removes a handoff: a single representation can serve as both the medium of perception and the target of synthesis, which is what makes edits addressable and observations usable as evidence. Unification by itself leaves the capability gaps above in place; it gathers them into one architecture. The sharper formulation is representational: whether the model carries an explicit spatial, temporal, or structural state that an edit can address, or whether each edit must re-sample a global latent wholesale.

Foundation-model progress in an agentic setting calls for a different measurement from a single-shot setting. A more informative question is which downstream agent decisions improve when the foundation model changes under a fixed controller and budget: fewer repair loops for the same edit, fewer misread observations during verification, fewer collapsed candidates during search. A weaker generator taxes the controller with repeated retries and inflates the planning burden; a weaker interpreter corrupts the agent’s task representation before any tool is invoked. Treating foundation-model improvement as one factor in a controlled agentic evaluation connects model progress to system progress.

13.2.2. Agent Architecture and Control

Agent architecture determines how task information, visual state, plans, tools, observations, and actions are organized, and the practical design space spans external orchestration over frozen models, native visual agents, hierarchical workflows, role-specialized teams, executable programs, and hybrids of these forms. The recurring structural failure across all of them is that a collection of components still behaves as a fixed pipeline: information is lost at each interface, a specialist optimizes a local objective that conflicts with the project, a coordinator repeats the same action after different failures, and the boundary between planning, execution, and correction is diffuse enough that responsibility for a failure is hard to localize.

The architectural question that matters is where decisions live and what state they share. Complex visual tasks require decisions at several levels—project organization, operation selection, local editing, verification, delivery—and these levels have different time constants, different information requirements, and different costs of being wrong. An architecture that funnels all of them through a single context window couples them unnecessarily; an architecture that isolates them into uncommunicating specialists loses the cross-level constraints that make the result coherent. The useful middle ground makes state transitions explicit: a shared project state that all roles read from and write to through typed interfaces, decision rights that specify which level can commit which kind of change, and evidence requirements that specify what an observation must contain to justify a transition. This resembles how mature software systems manage concurrency more than how current agents manage turns.

Two further directions follow from treating the architecture as a control problem. Adaptive depth allocates reasoning and search according to uncertainty and consequence, so routine edits take the fast path and ambiguous or risky decisions trigger deeper deliberation—the principle behind test-time compute scaling in reasoning models, applied here to the action loop. Reconfigurable roles treat agents and tools as replaceable components with declared preconditions and effects, so that a role can be created, merged, suspended, or substituted as the task demands, and a failure in one component can be isolated. Both require the architecture to know, at each step, what it is doing and why—a property that current systems approximate through prompts and rarely enforce through structure.

13.2.3. Visual Reasoning, World Knowledge, and Causal Understanding

Visual reasoning connects language, visual evidence, structured relations, external knowledge, and the space of possible actions; causal understanding adds a prediction of how an artifact or environment will change under an intervention. Current systems produce plausible appearances with incorrect relations, dimensions, event orders, and physical consequences, and the failure has practical weight: it reflects the gap between correlation and mechanism, in that a model that has seen falling objects has not necessarily internalized gravity. The empirical evidence is consistent across settings—generative models trained on physical dynamics generalize inside the training distribution and fail on out-of-distribution initial conditions, indicating case-based internalization of training examples.

For an agent, this gap determines whether the system can plan. Rolling out a candidate edit internally, predicting what a scene will look like after a motion, or forecasting how a diagram will break when a value changes all require an action-conditioned predictive model, and the properties such a model needs—causality, interactivity, persistent state over long horizons, physical accuracy—are largely the properties that current video generators lack. The research trajectory on world models makes the requirements explicit and the trade-offs measurable: causal autoregressive structure enables interaction, persistent memory enables coherence across an extended rollout, and physical fidelity trades against real-time responsiveness. Different applications weight these differently, but an agent that intends to plan in the world needs at least the first two.

Closing the gap between appearance and mechanism calls for representations that make structure explicit. Scene graphs, layouts, executable programs, geometric and simulation states, retrieval-augmented factual grounding, and explicit uncertainty each shift part of the reasoning burden out of the latent space into a form that can be inspected and verified. Training can reinforce the same shift: counterfactual supervision that varies an action while holding the initial state fixed teaches the model what depends on what, and evaluation that scores the relation between an output and the process that produced it rewards the same structure. The end state is a model for which “the result is correct” and “the result follows from the input the way the task intended” coincide, which is what the term visual reasoning should mean in an agentic setting.

13.2.4. Agentic Reinforcement Learning and Process Training

Process training teaches an agent how to make a sequence of decisions: which tool to call, how to construct a prompt, when to inspect, when to revise, and when to stop. Reinforcement learning connects those decisions to outcomes observed later in the trajectory. The central difficulty is credit assignment: a final-artifact score provides weak information about intermediate choices, a useful repair several steps earlier receives little signal, and a lucky sample can receive more than a deliberate one. Learned evaluators introduce a second difficulty—specification gaming—whose documented forms in agentic settings include exploiting the verifier, optimizing a proxy metric while damaging user intent, preserving an already-correct region that a metric happens to reward, and, in the sharpest cases, modifying the evaluation itself. The empirical pattern is consistent: the stronger the optimizer and the weaker the reward, the more reliably the gap between the metric and the intent is exploited.

Process rewards are the standard response and remain an open problem. Step-level supervision requires annotations or evaluators that can judge an intermediate decision, which is hard for visual work: whether a tool call was correct depends on information that becomes available only later, and whether an intermediate image is on track depends on the intended final state. Verifier-guided training and test-time search improve the signal but concentrate effort on the verifier, which then becomes both the bottleneck and the attack surface. A robust training recipe therefore combines signals with different failure modes—outcome rewards, process rewards, preference comparisons, environment feedback, counterfactual branches, failure injection—so that no single evaluator’s shortcuts dominate the learned policy.

Reward design for visual agents also has to encode the properties that make agentic creation distinct from single-shot generation: constraint satisfaction, collateral change to preserved regions, resource expenditure, willingness to ask for help, and the cost of an irreversible action taken under uncertainty. These belong among the primary objectives of an agent operating over a long trajectory, and a reward that omits them teaches a policy that succeeds on the benchmark and struggles in deployment. The research frontier lies in reward and verifier design that survives optimization, and in evaluation that can tell a policy that learned the task from one that learned the test.

13.2.5. Memory, Continual Learning, and Self-Evolution

Memory makes information from earlier operations available to later decisions; continual learning changes how the agent behaves across tasks; self-evolution creates or updates reusable strategies, skills, tools, and routing rules. The three are often discussed together and fail for different reasons. Memory in current systems is largely a retrieval problem over dialogue and artifacts, and its documented failure is state tracking: as facts, constraints, and decisions are revised over a long interaction, an agent that retrieves by similarity returns the superseded state as readily as the current one, because similarity to the query is a poor signal for what is true now. Long-horizon failure studies place memory limitations and catastrophic forgetting among the dominant categories, with a specific mechanism—an early constraint remains in the context window and is no longer attended to during generation—that persists even as context windows grow.

Continual learning across visual projects has an additional structural requirement: the experience has to be represented at a useful level of abstraction. A stored trajectory tied to one tool’s parameter names transfers poorly; a stored strategy with explicit preconditions transfers well. The reusable units of visual work are skills, repair patterns, tool compositions, and constraint templates, and the interesting questions are how to extract them from successful trajectories, how to attach the conditions under which they apply, and how to detect when an accumulated skill has become stale because the underlying tool or model changed. A skill library that lacks versioning reproduces yesterday’s workarounds against today’s interfaces and turns into a liability over time.

Self-evolution sharpens both problems because an update that goes unmeasured becomes a hidden change of policy. A benchmark gain that comes from extra retrieval or sampling while the underlying behavior stays fixed reflects test-time adaptation; a system that absorbs an error into its memory reproduces it across future tasks. The safeguards are the ones established in continual learning—held-out evaluation, forgetting and contamination checks, versioned updates with rollback—applied to behavioral change, which is the unit that matters when the learned artifact is a skill library or a router. The practical payoff for visual agents is concrete: fewer re-derived tool paths for repeated project types, personalization to a user’s stylistic commitments, and transfer of a repair pattern from one domain to another, all of which require memory that is structured, evaluated, and reversible.

13.2.6. Multi-Agent Collaboration and Human Co-Creation

Multi-agent collaboration divides interpretation, generation, critique, retrieval, editing, and delivery across several roles; human co-creation adds user goals, preferences, contextual knowledge, evaluation, and authorship to the same process. Multi-agent designs promise specialization and breadth; their recurring cost is coordination, which shows up as duplicated work, disagreement about the current state, optimization of incompatible local objectives, communication that rivals the compute budget of the generator itself, and error propagation through handoffs. Failure analyses of multi-agent systems repeatedly identify the same mechanism: without a shared, authoritative state, each role maintains its own picture of the project, and the divergence surfaces in the final artifact.

Human co-creation inherits the coordination problem and adds an asymmetry of visibility. A user whose visibility ends at the final output has no way to tell whether an important decision was made at the specification level, where their preference would have changed it, or in an intermediate step, where changing it now is expensive. The consequence is either micro-supervision, in which the user checks every low-level operation because the system does not escalate, or silent divergence, in which the system produces something polished but off-target. Both failure modes trace back to architecture: the current designs lack an explicit policy for when the system asks, when it proposes alternatives, when it defers, and when it proceeds—an adaptive initiative that tracks the risk and reversibility of the current decision.

The design levers are correspondingly structural. Shared state with typed ownership makes the current project state authoritative and reduces cross-role divergence. Explicit communication protocols bound the coordination cost. Role-specific evidence assigns each specialist the observations it is entitled to act on. Conflict resolution specifies what happens when two roles disagree. User-selectable levels of intervention let a person tune the granularity at which they participate, from high-level direction to local edits, with the system surfacing trade-offs alongside outcomes. Evaluation of such systems has to measure coordination cost, user workload, and authorship clarity alongside task quality, because a multi-agent architecture that improves output while making the process opaque has traded a visible problem for an invisible one.

13.2.7. Efficiency and Scalable Deployment

Efficient deployment concerns the quality, latency, compute, energy, and human time required to operate an agentic visual system. The cost structure of an agent differs from that of a single model call in ways that shape deployment: model calls, tool calls, retrieval, simulation, multi-agent communication, candidate generation, and review compound over a trajectory, and in visual work the candidates are large—video, 3D scenes, layered documents—so the marginal cost of one more try is far higher than in text. Multi-agent communication and visual verification can rival or exceed the cost of the generator itself, and a workflow that performs acceptably in a demonstration can become impractical for long videos, dense 3D scenes, or many concurrent users.

The principled response is to allocate compute according to uncertainty and consequence. Test-time scaling in visual generation has matured into an explicit trade-off between sample count, verifier quality, and final performance, and the empirical lesson parallels the one from reasoning models: verification-guided scaling is more sample-efficient than blind sampling, and a calibrated stopping rule recovers most of the quality at a fraction of the cost. The same logic extends to the action loop—a useful agent estimates the expected value of another action, selects the depth of verification, reuses cached results, and chooses between fast and slow paths—so that effort concentrates on the decisions where it changes the outcome. Fixed retry budgets remain the default in many systems and are the least developed part of this design space.

Deployment also constrains which architectural and learning choices are viable, and the constraint runs in both directions. A causally structured, few-step generation process makes interactive visual systems feasible and trades against the physical fidelity that high-stakes applications need, and different deployments weight the two differently. Serving infrastructure that treats generation, verification, and memory as independently optimized stages leaves the joint optimization unexploited; reporting quality in isolation hides the quality–cost and quality–latency trade-off that deployment actually faces. Energy, human review time, and the cost of rejected candidates belong in the same report as model expenditure, because for an agentic system they are of comparable magnitude and are currently the least visible part of the bill.

13.3. How to Establish Reliable Evaluation

13.3.1. Beyond Final-Artifact Quality

Final-artifact evaluation measures the appearance, semantic alignment, or task quality of the output; agent evaluation also measures how the system interpreted the task, chose actions, used evidence, handled resources, and involved people. The reason is attribution: a final score leaves open whether improvement came from the foundation model, the controller, extra samples, a human correction, or a lucky branch, and an agentic system that claims planning, reflection, memory, or recovery is making a claim about process behavior that a last-frame metric leaves unobserved. A fixed pipeline that retries the same call can earn the same score as a system that adapts its behavior, which makes the score uninformative about the claim under test.

Process evaluation has to be designed with intent. Which decisions the system faced, what evidence was available at each, what action was taken, what the alternative actions were, and what the action changed are the units of agentic behavior, and a report that separates artifact quality, constraint satisfaction, state retention, action selection, diagnosis accuracy, recovery, stopping, resource use, and human intervention can localize an improvement to the component that produced it. Attribution at this granularity turns an evaluation into a diagnostic instrument: it tells a researcher whether the next unit of effort should go into the foundation model, the controller, the memory, or the verifier. Baselines play a corresponding role—holding the generator, tool set, sampling budget, and human access fixed while varying the controller or learning method is what makes an agentic claim interpretable.

Practice in the field has started to move in this direction, with trajectory-grounded automatic judges used to attribute failures to categories such as planning error, memory limitation, or constraint loss, and with the composition of failure types shifting measurably as task horizon grows. Extending this practice to visual agents requires solving the problems specific to visual trajectories: observations that take the form of images, artifact versions that must be tracked across edits, and intermediate states that lack a compact symbolic form. A process evaluation for visual agents is therefore also a data-engineering problem—what to record, at what fidelity, and how to make a long multimodal trajectory comparable across systems—and the evaluation claims below draw on its solution.

13.3.2. Process, Causal, and Counterfactual Evaluation

Process evaluation records the sequence of states, observations, actions, results, and decisions. Causal evaluation asks whether an action produced the observed improvement. Counterfactual evaluation changes an input, an observation, or an available action and tests whether the behavior responds. The three are layered, and current practice sits mostly below the first: reported trajectories routinely omit the observation used at a decision point, the version of a tool, the reason for a retry, or the state before a correction, which leaves open whether feedback changed the action or whether the system would have produced the same output on a fixed path. The defining property of an agentic loop lies in the connection between evidence and action, and a benchmark that leaves this connection unexposed has limited power to separate adaptive control from rehearsed execution.

The concrete design instrument is perturbation. Injecting localized defects, unavailable tools, changed references, invalid programs, conflicting constraints, or misleading observations measures what a system does when the world departs from its expectation, which is the situation an agent is built for. The measurable responses are diagnostic: whether the defect is detected, whether the diagnosis identifies the right cause, whether the repair is local or destructive, whether preserved constraints survive, how long recovery takes, and whether the system stops or escalates appropriately. Comparing the action selected under alternative evidence isolates the causal contribution of the observation to the decision and provides direct evidence on whether the loop is closed.

For visual agents, perturbation has an additional dimension because observations are high-dimensional and generative. A misleading screenshot, an ambiguous reference image, a partially corrupted intermediate artifact, or an adversarial instruction embedded in retrieved content are all realistic observations, and clean-input evaluation leaves the agentic behavior itself largely untested. Benchmarks that treat visual perturbation as a first-class axis, alongside tool and task perturbation, are the corresponding open need, and the payoff extends past robustness: the same machinery that measures recovery from an injected defect measures whether a claimed reflection or self-correction mechanism actually modifies behavior, which is the claim most often made and least often tested in current systems.

13.3.3. Long-Horizon and Cross-Task Evaluation

Long-horizon evaluation tests a system over many related decisions; cross-task evaluation tests whether information from earlier tasks affects later ones under new prompts, artifacts, tools, users, or domains. Both are under-served for the same reason: they require evaluation infrastructure that maintains state across episodes, while most benchmarks are collections of independent single-step instances. The empirical consequence is well characterized in adjacent domains—short episodes conceal state drift, memory errors, repeated failures, and stopping problems, and performance degrades as the number of interdependent decisions grows, with planning-related and memory-related failures dominating the composition of errors at longer horizons. Visual creation adds drift modes of its own—identity, layout, style, and reference drift—that accumulate quietly until the artifact is visibly inconsistent with its own earlier state.

Cross-task evaluation carries a sharper trap because self-improvement claims are easy to state and hard to evidence. An update derived from a set of tasks and evaluated on the same distribution shows in-distribution adaptation; evidence for transfer requires held-out tasks. A memory that retrieves the most similar past case can improve a benchmark through lookup while leaving the policy unchanged. Meaningful evidence requires separating formation tasks, development tasks, and held-out transfer tasks, and introducing shifts that the update could not have anticipated: changed requirements, new tools, new visual domains, delayed references. The measurable quantities are then the ones continual learning has standardized—forgetting of prior competence, contamination of the evaluation, persistence of the update, and the relation between a specific stored experience and a specific later decision.

Establishing this evaluation for AVG bears directly on the L5 claims that the autonomy scale in this survey defines. Behavior that changes under experience is difficult to validate with a static test set; the test set has to evolve with the system or be partitioned against it. Long-horizon and cross-task benchmarks for visual agents therefore carry significance well past an expanded evaluation suite: they bear on whether the upper end of the autonomy scale describes current systems or anticipates them.

13.3.4. Multidimensional Visual Correctness

Visual correctness spans several partially independent axes, and these axes can conflict. Appearance, semantic alignment, spatial relations, geometry, topology, text, data, identity, temporal ordering, physical plausibility, executable behavior, and edit preservation can each fail on its own, and improving one can degrade another—a more attractive render can break a dimension, a stricter preservation can block a needed repair. A single score compresses these failures and, more consequentially, obscures which requirement failed, which is the information an agent uses to plan a repair. A visually convincing result with an incorrect value, a missing object, an impossible shadow, or a changed identity is precisely the failure mode that appearance-weighted evaluation rewards and that domain users cannot accept.

Different artifacts require different evidence sources. An image, a CAD model, a chart, a page, and an interactive interface each expose correctness through a different channel—rendering, geometric validation, data re-execution, parsing, and interactive testing respectively—and a single visual-similarity metric measures appearance while leaving the other channels largely unexamined. The corresponding evaluation architecture represents requirements as typed claims and associates each claim with the check that can adjudicate it: learned judgments where appearance is the claim, symbolic and programmatic checks where structure or data is, execution and simulation where behavior is, and human review where acceptance is. Preserving per-claim outcomes and confidence alongside any aggregate keeps the diagnostic information that a single number discards.

For an agent, multidimensional correctness has a further role: it is the interface between evaluation and control. A verifier that returns a typed claim—what failed, where, and with what confidence—is actionable, whereas a scalar score forces the controller to guess what to change. Closing the loop between multidimensional evaluation and repair is one of the higher-leverage connections in the field: the same typed-claim representation serves the verifier that guides sampling, the diagnostic that localizes a defect, and the report that tells a user what to trust, so progress here compounds across the stack.

13.3.5. Human, Cost, Safety, and Provenance Evaluation

Human, cost, safety, and provenance evaluation measures the conditions under which a visual agent can be trusted and used: user understanding, intervention timing, workload, resource expenditure, untrusted inputs, tool side effects, privacy, authorship, and the history of an artifact. Current reporting under-measures these factors. User studies report preference without control or workload; system reports omit human corrections and the cost of rejected candidates; provenance is lost through editing, delegation, format conversion, and rollback. The result is an evaluation regime in which the cost of producing a deliverable and the conditions under which a person could trust it both stay out of view.

The safety surface of a visual agent is wider than that of a text agent because the input channel is wider. Visual and audio content can carry instructions that the user did not authorize—embedded in retrieved images, rendered pages, documents, or screenshots—and multimodal injection is a documented, reproducible attack against the vision-language backbone that most visual agents sit on. An agent that treats each observation as trusted data can be steered by a single injected image; defenses therefore have to be built into the architecture, separating sources of authority from sources of content, and evaluated under adversarial testing. Provenance and content-credential machinery gives the corresponding record on the output side—what was generated, from what sources, edited by whom—and matters for an agent in particular because its artifacts accumulate edits across a trajectory and the provenance record has to survive that accumulation.

The unifying frame is calibrated trust: what matters is the relationship between system ability, human oversight, and consequence, and a quality score gives limited evidence about it. The measurable quantities are the ones that make that relationship visible: reporting model, tool, retrieval, simulation, communication, and human costs as separate line items; testing user understanding and intervention timing alongside preference; probing permission boundaries, prompt injection, and reversible action coverage as first-class cases; and connecting sources, transformations, responsible actors, and approvals across the full trajectory. These measures bear directly on whether a visual agent can be deployed where it matters, and they are currently the least standardized part of the evaluation stack.

13.4. More Questions About Agentic Visual Generation

13.4.1. What Does Intelligence Mean in Visual Creation?

Intelligence in visual creation includes understanding, representation, generation, action selection, adaptation, exploration, collaboration, uncertainty management, and responsibility, and the relative value of these capacities depends on the task. A system can be strong in appearance and weak in structural reasoning, strong in local editing and weak in project continuity, strong in exploration and weak in user control, and a single ranking obscures these differences. The same divergence appears across applications: personal artwork, scientific figures, engineering designs, educational media, and high-consequence applications weight creativity, accuracy, speed, autonomy, and oversight differently, so systems optimized for different profiles occupy different points in a capability space.

The practical implication is that intelligence claims are best stated as profiles under explicit conditions: which capacity improved, under what task and budget, at what cost to other capacities, and with what effect on the user and the artifact. Profile reporting makes comparison meaningful across settings that value different things, and it raises the question that aggregate scores set aside: intelligence for whom, and for what. For AVG in particular, where the artifact carries intent and the process carries control, an intelligence claim needs an account of what the system understood, what it changed, and what it preserved before it can be evaluated.

13.4.2. Authorship, Responsibility, and Human Control

Agentic creation distributes creative decisions across users, agents, foundation models, tools, data sources, and evaluators, and the distribution raises two questions that a quality metric leaves aside. Authorship concerns contribution and creative direction—who shaped the intent, who selected among alternatives, whose style the result carries. Responsibility concerns who can approve, explain, change, or reverse a consequential decision, and it becomes acute when an agent’s output affects external files, public content, collaborative projects, or regulated activity. Multiple models and tools contribute to a single artifact in current systems, and their individual contributions and permissions are difficult to reconstruct after the fact, which makes both questions hard to answer even when the process was well intentioned.

The design goal is meaningful control at the points where values enter the artifact. Editable intermediate states let a user intervene while intent is still structural, before it is baked into pixels. Approval checkpoints attach to consequential and irreversible decisions. Contribution records and provenance connect an output to the sources, models, and decisions that produced it. Permission boundaries and reversible actions define what the system may do unilaterally and what it must escalate. Adaptive initiative ties the level of autonomy to the risk of the decision. Together these mechanisms maintain authorship and responsibility during operation, at the stages where they can still be influenced.

13.4.3. From Individual Models to Creative Ecosystems

A creative ecosystem combines foundation models, agent policies, tools, data, memory, evaluation services, interfaces, human workflows, and deployment infrastructure, and the complete system determines what a user can create and how reliably a result can be revised and delivered. Ecosystem-level fragility is already visible: a workflow can depend on a particular model version, external API, evaluator, data source, or application interface, and a change in one layer alters the behavior of the whole pipeline while every local component still works. Communication overhead, licensing, dependency drift, and attribution become the binding constraints as the ecosystem grows, and single-model improvements leave them in place.

Ecosystem-level design is a research object in its own right. Model and tool contracts specify behavior at interfaces so that one component can be replaced without re-deriving the workflow. Version dependencies and provenance records make the behavior of a deployed system reproducible and auditable. Evaluator behavior specified as part of the contract keeps a silent verifier change from invalidating a trained policy. Permission boundaries and resource budgets travel with the workflow. Component substitution that preserves the meaning of the artifact and the trajectory record extends to the ecosystem level the locality that editing seeks at the artifact level. Visual creation is moving toward reusable workflows and shared services, and progress in any single model or tool compounds across many applications in proportion to the stability of these interfaces, provenance mechanisms, and evaluation protocols.

The frontier agenda across all three paths therefore concerns a changing unit of visual intelligence: a single result, a multimodal workflow, a sustained creation process, a system that learns from its history, and a responsible participant in human creative work. Each stage requires evidence about what the system understood, what it changed, what it learned, what it cost, and how people can control its consequences, and each stage adds to the obligations of the previous one.

14. Conclusions

This survey studied agentic visual generation as goal-driven visual creation in which runtime evidence changes subsequent creation decisions. We defined and formalized this behavioral criterion, distinguished AVG from single-pass generation and fixed workflows, and introduced the L1–L5 levels to describe state-dependent control and cross-task improvement. We also proposed a six-component analytical framework comprising Goal Understanding, Specification, and Planning; Memory; Tool; Perception; Action; and Cross-Task Self-Improvement.

Using this framework, we compared agentic visual systems across image generation and editing, video and animation, three-dimensional and CAD creation, scientific visualization, structured documents, user interfaces, Web artifacts, and cross-domain workflows. We further reviewed training targets, trajectory supervision, policy initialization, multimodal feedback, reinforcement learning, experience reuse, skill abstraction, and transfer. The evaluation chapter organized evidence into artifact quality, goal and constraint satisfaction, trajectory and decision quality, and system and human-centered evaluation, while the frontier chapter summarized the major directions for expanding AVG scope, improving visual-agent intelligence, and establishing reliable evaluation.

Future work can extend this analysis to broader and more advanced agentic visual systems, including multimodal, embodied, multi-agent, and continually improving agents operating across longer tasks and more diverse environments.