Measuring Frontier Code Agents at Digital-Twin Scene Construction

1. Introduction

Simulation-based policy learning and evaluation for robot manipulation still begins with a manual step: constructing a digital twin, or at least a visually matched approximation, of the deployment scene by human modeling, scanning, or annotation of the site (Li et al. 2024b; Torne et al. 2024). The cost repeats with every deployment site, since each site requires its own in-distribution scene, and policy performance degrades under variation in object and table textures and camera placement (Li et al. 2024b), and under lighting and clutter changes (Pumacay et al. 2024). Prior work automates twin construction with purpose-built systems of three kinds. Reconstruction pipelines recover a scene from a photograph through a fixed perception chain of object detection, depth estimation, asset retrieval or generation, and pose estimation, or reconstruct individual objects into simulation-ready assets (Cao et al. 2026; Dai et al. 2024; Gu et al. 2026; Le et al. 2025; Zhao et al. 2025). Language-model composers assemble scenes from asset libraries, prompted by text rather than a photograph (Wang et al. 2024b; Yang et al. 2024). Agentic builders wrap a language model in a custom scaffold of roles, skills, and verification loops that steers it through scene construction. None of these systems measures what an unmodified general-purpose code agent achieves on the same task, with no custom harness built around it. The hypothesis of this paper is that purpose-built harnesses arose to compensate for capabilities that earlier models lacked, and that the gap they compensate for narrows with every frontier model and their agent harness release.

This paper measures the gap that remains between an unmodified code agent and purpose-built pipelines at scene reconstruction, a setting referred to here as Frontier Agent as Scene Constructor (FASC), by analogy with FAEA (Tsui et al. 2026). FASC names the setting rather than a method: the agent is used as shipped, and the only inputs are the photograph, the simulator, and one prompt. Seven frontier code agents, each used as shipped through Claude Code or OpenAI Codex, rebuild eight BEHAVIOR-1K rooms from one photograph each and are scored against the rooms’ known 3D layout on ACDC’s sim-to-sim protocol (Dai et al. 2024), with memorisation and file-access contamination controlled (Figure 1; Section 3). From the photograph the agent estimates the object layout (and, when no calibrated camera is supplied, the camera pose), then selects, places, and configures assets from a catalog. The two strongest of them, Codex with GPT-6 Astra and Claude Code with Opus 5, are then tested for generality on farm photographs, a domain with no calibrated camera and no 3D reference, where each agent places plants pre-grown with a procedural botanical model and each twin is validated against its photograph, and where the two perform comparably (Section 5). On the rooms Astra places objects within 14 cm of ground truth and retrieves the exact asset model for 66% of the scored objects. The gain from one model generation to the next is large within each provider as well. Claude Opus 5 places objects within 39 cm, a 4× reduction in placement error over Opus 4.6, released under six months earlier, whose reconstructions recover too few objects to be practical. Within Codex, GPT-5.6 Sol cuts GPT-5.5’s placement error by a third and nearly triples its exact-asset recovery, and GPT-6 Astra, released eight weeks after GPT-5.6 Sol, cuts its placement error from 76 to 14 cm and doubles its exact-asset recovery, suggesting that reconstruction performance scales with frontier-model capability independent of harness engineering.

2. Related Work

Twins from images. Simulation-ready scenes have been constructed in several distinct ways. Most scenes and assets are still modeled manually (Jin et al. 2026b; Khanna et al. 2024; Kolve et al. 2017; Li et al. 2024a; Szot et al. 2021); language-guided systems compose them from asset libraries without a photograph (Wang et al. 2024b; Yang et al. 2024); and real-to-sim pipelines rebuild one specific real setup, by manually matching a simulated scene’s textures and background to the real workspace (Li et al. 2024b), by scanning the environment and manually annotating the joints of its articulated objects, such as drawers and cabinets (Torne et al. 2024), or by fitting the physical parameters of deformable objects to video of a person manipulating them (Jiang et al. 2025; Zhang et al. 2026b). A growing line of work instead turns visual observation directly into simulation content: recovering geometry, physics, and camera pose from one image (Zhao et al. 2025), articulating assets (Cao et al. 2026; Chen et al. 2024; He et al. 2025; Le et al. 2025; Wu et al. 2026b), and composing scenes from images and panoramas (Lee et al. 2026b; Lu et al. 2026a; Wu et al. 2026a; Zhang et al. 2026c) or video (Dan et al. 2025; Han et al. 2026; Jain et al. 2026; Wang et al. 2026c; Xia et al. 2026); IGen (Gu et al. 2026) likewise lifts a single image into a point-cloud workspace but renders observations from point clouds rather than a physics-based reconstruction. ACDC composes cousin scenes from one image (Dai et al. 2024), and PRISM generates demonstrations from an image and an instruction (Ko et al. 2026). Both vary the object instances rather than reproduce the photographed site, and both retrieve assets rather than generate them. SimFoundry and GRS instead require a video scan or depth sensing (Ranawaka et al. 2026; Zook et al. 2025). Concurrent work reaches simulator-usable scenes from real observations through purpose-built pipelines. Lucida parses multi-view video into a scene graph, generates an asset for each instance and places it with a trained placement policy (Qin et al. 2026), and NeoWorld-Pro has a multimodal model write scene programs from a single image and refines them with physics-in-the-loop validation (He et al. 2026). Each pipeline fixes its strategy—reconstruct, retrieve, or generate—at design time.

Agentic scene builders. A newer family wraps the model in role or skill harnesses (Chen et al. 2026; Pfaff et al. 2026; Yang et al. 2026). Several build from a description rather than a photograph (Ning et al. 2026; Wang et al. 2026a,b), and those that take an image score 3D geometry only against synthetic ground truth or pseudo-labelled scans (Kim et al. 2026; Liu et al. 2025), or report no 3D-ground-truth metric (Kang et al. 2026). The nearest, SceneConductor, orchestrates specialized agent roles across a three-stage pipeline from a single image to a Blender scene (Kim et al. 2026), and EmbodiedGen V2 connects off-the-shelf coding agents to a skill harness with policy training in its generated worlds (Wang et al. 2026b). Code agents in robot pipelines are wrapped the same way, calling authored primitives, writing task and reward code, or generating integration glue inside designer-built loops (Berman et al. 2026; Chen et al. 2025; Elmaaroufi et al. 2026; Fu et al. 2026; Hua et al. 2024; Jin et al. 2026a; Jing et al. 2025; Katara et al. 2024; Lee et al. 2026a; Li et al. 2026; Liang et al. 2023; Liu et al. 2026; Lu et al. 2026b; Ma et al. 2024; Singh et al. 2023; Wang et al. 2024a,b; Xiao et al. 2026; Zhang et al. 2026a); unmodified agents have acted from privileged state or written algorithmic policies (Tsui et al. 2026; Xie et al. 2026), in neither case referenced to a real scene. This work instead gives one unmodified code agent the simulator and domain-specific generators as tools, so retrieval, generation, and reconstruction are choices the agent makes. It recovers the object layout from the photograph, and the camera pose when none is given, then selects, places, and configures assets. Each twin is validated against 3D references when available and against the photograph otherwise.

3. Reconstructing Each B1K Room from One Photograph

This paper first tests whether one unmodified code agent, given a photograph, references the simulator library and composes a physical 3D twin of the photographed scene. BEHAVIOR-1K (B1K) rooms with known 3D ground truth are scored on ACDC’s sim-to-sim protocol (Dai et al. 2024) (Figure 1); in this paper’s setup the camera pose is given and every object must be retrieved from the B1K asset catalog (departures from ACDC’s published setup are listed in Appendix A). ACDC is the reference because it publishes scene-reconstruction fidelity under a defined protocol; the nearest comparable system, PRISM, reports policy success rather than reconstruction fidelity and had released no code at the time of writing (Ko et al. 2026).

In this experiment, each agent rebuilds one B1K room (Li et al. 2024a) from one photograph of it, and the reconstruction is measured against the room’s known 3D layout.

Agents and the task. Seven code agents run the same task: Claude Opus 5, Claude Fable 5.1, Claude Fable 5 and Claude Opus 4.6 through the Claude Agent SDK, the programmatic harness of Claude Code, and GPT-6 Astra, GPT-5.6 Sol and GPT-5.5 through OpenAI Codex. The SDK is used rather than the interactive client so that every run’s transcript is persisted and can be audited for instruction adherence and potential cheating (Appendix C). Each agent is used as shipped with default tools. Ground truth can leak through two channels, and both are controlled. First, the model’s weights: B1K is public, so a model may have memorised its rooms. Rooms are therefore referred to by obscure ids only. A recitation probe, which gives the model a room’s real B1K name with no image and no tools and asks it to list the room’s assets, scores zero on every reported room for Opus 5 and for GPT-6 Astra; for Fable 5.1 it names one scored asset in each of two rooms, both assets that agents with a zero probe also retrieved from the photograph (Appendix C). Second, the filesystem: the ground-truth files sit beside the workspace. During a session these files, the library’s scene directories, and every other run’s directory are unreadable, and the agent’s web tools are disabled. Afterwards, every tool input in the transcript, sub-agents included, is searched for attempts to read them. Seven reported runs made such attempts, all denied, and one further run (Fable 5.1, room J) read the benchmark’s own README, a file outside the lock that holds no room data, and is kept with the audit flag it received (Appendix C); none obtained ground-truth content. In every arm the whole task is the following prompt:

Input: scene.png in this directory is a photo of one room built from BEHAVIOR-1K assets. The asset catalog is catalog.jsonl (one line per model: category, model, bbox_size in metres). render.py is an OmniGibson example that renders a scene json from the photo’s camera ('source env.sh && og_py render.py --pred pred.json', about a minute; the json format is in its header). Modify it or write your own scripts as needed: walls, floor, lighting and anything else in the background are yours to build; furniture must come from the catalog. cam.json holds that camera: {{CAMERA}}

Code up a digital twin of the room as pred.json.

Regenerate and rerender until reviewer agent approves for similarity. Reviewer compares similar objects and scene background and make sure to use the most similar ones, also Check whether object detected is indeed sensible for the placement

Do not read anything under behavior-1k-assets/scenes.

The scene file the agent writes lists each asset’s position, orientation and size. The prompt instructs the agent to revise until a reviewer agent approves, but no reviewer prompt is provided: the agent must create the reviewer itself, as a sub-agent whose instructions and model it chooses. In every arm the session ends only when the agent ends it, since no turn, time or cost cap is imposed. The settings of each agent arm and the room list are given in Appendix C, and Appendix B compares Opus 5 runs on five rooms with and without the prompt sentence “Modify it or write your own scripts as needed”.

Results. ACDC’s exact evaluation rooms were never released, so its published main-paper table is quoted as is. The agents are scored on eight rooms assembled here, a bedroom, a living room, an office, a kitchen, a dining room, a bar and two bathrooms, spanning 6–14 scored objects and 3.5–9.0 m. Room selection, the camera procedure and the scoring rules are given in Appendix C.

GPT-6 Astra performs best of the seven agents on the eight rooms, no run having obtained ground-truth content (Table 3 in Appendix C, with ACDC’s published values (Dai et al. 2024) for reference). It assigns the correct category to 72% of the scored objects and retrieves the exact asset model for 66% of them. Its placement error, the distance between the centre of a placed object and the centre of its matched ground-truth object, is 14 cm in the median run. It is the best agent on 5 of the 8 rooms by placement error and on 5 of the 8 by centred IoU. GPT-6 Astra places fewer scored objects than the rooms contain (123 of 150); an unplaced object counts as wrong for category, model and both IoU metrics but does not enter placement error. A gap to ACDC’s curated pipeline, whose main-paper table reports a median placement error of 6 cm on its four rooms, remains. Figure 2 shows the input photograph beside each agent’s final render for the best, median and worst room by mean placement error, and Figure 6 in Appendix C shows all eight rooms.

More strikingly, reconstruction performance improves substantially from one model generation to the next within each harness provider (Figure 3, top). Opus 4.6, released under six months before Opus 5, retrieves the exact asset model for 6% of the scored objects where Opus 5 retrieves it for 52%, and Opus 4.6 median-run placement error is 153 cm where Opus 5’s is 39 cm. In the Codex harness, GPT-5.6 Sol retrieves the exact asset model for 32% of the scored objects where GPT-5.5 retrieves it for 11%, and GPT-5.6 Sol median-run placement error of 76 cm is a third lower than GPT-5.5’s 120 cm. The same direction holds at the next Codex step: GPT-6 Astra, released eight weeks after GPT-5.6 Sol, retrieves the exact asset model for 66% of the scored objects where GPT-5.6 Sol retrieves it for 32% and places objects within 14 cm in the median run where GPT-5.6 Sol places them within 76 cm. Two caveats apply across all agents. The protocol does not penalise extra objects, and GPT-5.5 places nearly twice as many scored-category objects as the rooms contain. Orientation is recovered poorly by every agent, at about one radian of raw error, and ACDC’s 0.08 rad is reported after a symmetry restriction it does not define, so the two values are not comparable.

GPT-6 Astra gives the highest performance per unit cost of the seven agents and Fable 5.1 the highest of the four Claude agents, at median costs of $9 and $13 per room against $120 for Opus 5, which reaches its performance through far more work per room, 335 turns and four reviewer sub-agents at the median where Fable 5.1 takes about 90 turns and spawns one (Figure 3, bottom). Codex costs are list-price estimates, since those runs were billed to a subscription. Discussions in online forums such as Reddit also report that Opus 5 tends to use more thinking turns, but the why remains a mystery.

4. Code Reasoning as a Generic Harness

Section 3 and Section 5 establish what the agents build; this section examines how. FAEA analysed the traces of a code agent writing manipulation trajectories (Tsui et al. 2026). The same analysis, applied to every Opus 5 room run of Section 3, finds the same loop—write a script, execute it, inspect the output, revise—retargeted from trajectories to scenes with no change to the harness.

Tool orchestration. The 16 runs, two per room, contain 7,690 tool calls. Shell execution dominates at 65.0%, file reading follows at 32.6%, and the remaining tools share 2.4%; each run spawns 3.9 reviewer sub-agents and launches the simulator 56.4 times on average. The counts describe a write–render–inspect cycle, and their shape matches the Bash-dominant distribution FAEA reports on manipulation benchmarks, with one departure: where FAEA’s agent created scripts with the dedicated file-writing tool, this agent creates them from within shell commands, so file writes are counted under shell execution rather than under the file-writing tools. Appendix G reports the per-tool counts.

Analytic bootstrap. A recurring first step in the traces is analytic rather than iterative: before placing any object, the agent writes a projection script from the supplied camera and back-projects image edges into world coordinates. The main agent’s final summary in the room J round-1 run reads as follows (transcript quotes are verbatim, with code spans set in typewriter type):

“every number in layout.py comes from back-projecting image edges through cam.json — leg contacts on the floor plane gave the partition plane x = -6.50 ; the wall/floor line gave y = -47.87 ; the tile top edge gave z = 2.174 ”

Wall planes, footprints and heights enter the scene file as solved quantities, and iteration then corrects what the analytic solution does not reach.

Hypothesis-driven revision. When a render disagrees with the photograph, the agent proceeds as FAEA describes: it measures the mismatch, states a cause, and tests the cause, rather than perturbing coordinates. A reviewer sub-agent in the room F round-1 run, judging a mid-iteration render of a panel whose silhouette would not fit, writes:

“Unprojecting the photo’s edge onto horizontal planes gives x = −4.96 @ y=4.8 down to −4.11 @ y=0.1 — i.e. it is a raking eave , not a flat panel cut at constant x, which is why LATH_X = -4.55 cannot reproduce it.”

The hypothesis names the geometry and explains why the current constant cannot fit it; the fix follows from the measurement. The same procedure produces tooling: one run diagnoses that asset origins are offset from bounding-box centres and writes a calibration script that corrects each position against the renderer’s reported bounding boxes until the rendered box matches the target.

5. Extending to a New Domain: Farm Twins

In the room experiment, the camera pose is supplied and reconstruction fidelity is scored against known 3D ground truth. The question here is whether the two strongest of the same unmodified agents, Codex with GPT-6 Astra and Claude Code with Opus 5, extend to a domain with neither: farm fields with no calibrated camera and no 3D reference. Each agent must instead read the photograph—horizon, row geometry, plant habit, scale cues—and author the scene from it. Specialty-crop farming is chosen as the domain because it carries major sustainability and economic impact and still faces major automation challenges (Bac et al. 2014; Vougioukas 2019). A photograph is also the input a grower can supply with a phone camera, without specialized instruments or calibration, which makes it the natural entry point for a farm twin. Procedural crop-field generators such as CropCraft (Bertoglio et al. 2025) assemble randomized fields from a configuration file that no photograph constrains, and language models have been used to author agricultural simulation scenes from text (Yoncalik et al. 2026). Recovering procedural plant descriptions from imagery has been studied directly, by training a vision-language model to emit plant-architecture representations from simulated images and by prompting foundation models to fill plant simulation configurations in context (Yun et al. 2026a,b). Here each agent instead places the generator’s plants and composes the whole scene against the photograph.

The test involves 46 twins from each agent across ten crops, apple, strawberry, lettuce, broccoli, blueberry, table grape, peach, sweet cherry, tomato and sweet potato, five photographs per crop except broccoli with two and blueberry with four, each built from one online photograph and validated against it (Figure 4). The 46 photographs are openly licensed images from Wikimedia Commons, Flickr and rawpixel. The first is a canonical example of harvest automation in agriculture (Silwal et al. 2017; Zhang et al. 2024). The other nine are among the most labor-intensive fruit and vegetable crops per acre in national data (Martin 2020).

Whereas the room experiment isolates the agent for controlled measurement, the farm experiment measures how much each agent automates from a single top-level prompt covering all the photographs, mimicking how a real user would ask the agent. In this testbed, both agents have access to Isaac Sim (NVIDIA 2026b), a procedural plant generator, Helios (Bailey 2019), and a library of supporting assets absent from the room testbed. The library provides plant models for each crop, sky domes covering clear to overcast conditions, and tileable ground materials. The plant models are grown with Helios, using the nearest species in its library for the four crops it lacks. The sky domes come from the openly licensed Poly Haven collection (Poly Haven 2026). The ground materials are grass from the NVIDIA Omniverse Base Materials library (NVIDIA 2026a), with leafy grass and brown mud soil from Poly Haven. The prompt is reproduced in Appendix D. Each agent then wrote its own generator and reviewer sub-agent prompts and ran them.

During the iterations, each agent estimates and corrects the camera itself from the horizon and the crop rows’ vanishing point (Figure 5, with camera solutions and correction traces for the first eight Opus 5 runs in Appendix D), places plants pre-grown with the procedural plant-architecture model of the Helios framework (Bailey 2025) and fits their scale, spacing and colour to the photograph (plant-configuration traces in Appendix D), and authors the remaining objects as primitives at photo-measured dimensions (structure traces in Appendix D). Validation is render-to-photograph similarity under pretrained vision encoders (Oquab et al. 2024; Radford et al. 2021), ranked among all the twins’ reference photographs (the metric is specified in Appendix E).

Astra and Opus 5 are comparable at constructing these novel scenes. Under DINOv2, 24 of the 46 Astra renders and 22 of the 46 Opus 5 renders rank their own photograph first among the 46 references, the mean similarity of matched render–photograph pairs is 0.52 and 0.57 against 0.20 for mismatched pairs, and reference photographs of the same crop average 0.49, so both matched-pair means sit above the level that real photographs of the same crop reach against each other. Under the same encoder, 30 of the 46 twins of each agent rank their own photograph first among the two to five photographs of their own crop, so the embedding discriminates between scenes of one crop and not only between crops. CLIP compresses the range, with matched pairs at 0.76 for Astra and 0.80 for Opus 5 against 0.67 and 0.69 in the same order for mismatched pairs and same-crop photographs at 0.79, and ranks the own photograph first for 22 of the Astra renders and 25 of the Opus 5 renders. Neither agent ranks more twins first under both encoders. Astra built its 46 twins in one Codex session of two hours, against over 23 hours across two campaigns for Opus 5. The weakest twin under DINOv2 by own-photograph similarity is the fifth peach twin for Astra and the third sweet potato twin for Opus 5, and CLIP ranks those photographs seventh and first, so the two encoders disagree on which twin is weakest; per-twin values are in Table 5.

6. Task Formalization

The internals of Claude Code and Codex, and the way the frontier models behind them are trained, are not public, so nothing in this section describes how these systems are built. The formalization below is a mental model of the observed behaviour, assembled from the run transcripts of Section 3 and Section 5 and from the structure of open-source code agents.

The FASC setting of Section 3 and Section 5 shares one structure, stated here formally: scene construction as iterative program synthesis, following the framework of FAEA (Tsui et al. 2026). Given a task instruction ℓ, a reference photograph I, and a tool set $\mathcal{T}$ (the agent’s shipped file, shell, and sub-agent tools, through which it invokes a renderer), the agent generates a sequence of script attempts $\{{\sigma }_1,{\sigma }_2,\dots ,{\sigma }_N\}$. Each attempt ${\sigma }_i$ is a sequence of tool invocations that writes or revises a scene program, whose output is a scene file listing each object’s asset, position, orientation, and size. The agent maintains accumulated context

$${\mathcal{C}}_i=\left\{({\sigma }_j,o_j)\mid j<i\right\},$$

(1)

where $o_j$ denotes the observations from attempt j: the render, error messages, and any reviewer verdict. Each new attempt is generated conditioned on instruction, photograph, tools, and history,

$${\sigma }_i\sim \mathrm{LLM}(\ell ,I,\mathcal{T},{\mathcal{C}}_i).$$

(2)

Two departures from FAEA follow from the task. First, an attempt produces a scene rather than a trajectory: where FAEA’s attempts end in a terminal simulator state, an attempt here ends in a render $R\left({\sigma }_i\right)$ of its scene file. Second, the evaluator splits in two. FAEA’s evaluator $\mathcal{E}$ returns success inside the loop; here the in-loop judgment comes from a reviewer the agent itself creates and instructs—an internal evaluator comparing $R\left({\sigma }_i\right)$ to I—while the external evaluator $\mathcal{E}$ is withheld until the session ends. On rooms, $\mathcal{E}$ is ACDC’s metric suite against the room’s known 3D layout (Section 3); on farms, where no 3D reference exists, it is encoder similarity between the final render and the photograph (Section 5). The agent never observes $\mathcal{E}$, so a run terminates only when the internal reviewer approves or the agent ends its session; no gradient updates occur.

7. Conclusion

This paper measured what unmodified frontier code agents achieve at digital-twin scene construction from one photograph, the FASC setting, on eight BEHAVIOR-1K rooms with 3D ground truth and 46 farm photographs without it. Performance rises steeply from one frontier model generation to the next within each harness provider, which is consistent with the hypothesis that purpose-built harnesses and specialized models compensated for capability that newer general models now supply. The twins are certified on geometry and appearance, not on dynamics or on policy transfer.

Language processing moved from specialised models to a single large language model, and a capable general agent emerged on top of it. FAEA (Tsui et al. 2026) and ENPIRE (Xiao et al. 2026) showed that generic frontier agents can produce embodied trajectories and autonomously improve manipulation policies, and this paper shows that the same class of agent can reconstruct scenes. Multimodal models are taking on more of the tasks that only specialised models or tools could perform. A generic frontier agent built on top of them, accessible as an ordinary program such as Claude Code or Codex, can be expected to bring that intelligence, still largely confined to text today, into embodied settings. The emergent physics understanding capabilities reported for large video models (Wiedemer et al. 2025) point the same way. The trend indicates that embodied capability will keep improving while requiring less embodiment-specific engineering.

This paper has several limitations. The training of the latest frontier models is opaque, so for the models used here it is not clear whether this behaviour emerged from progress in the underlying model or was specifically trained for. The sample sizes are limited by available resources.

Reproducibility statement: The code and its execution environment are attached as supplementary material. Nonetheless, readers are encouraged to run their own code agent with a standard Isaac Sim installation and ask it, in whatever wording they like, with a photo, to create a digital twin. A native agent setup adapted without changes on two friends’ machines.

Use of Artificial Intelligence: This paper results from many conversations between the author and Claude Code with a Max 20X subscription. The author initially drafted the idea. Each step iterated between a token-heavy agent team and manual review. The steps span experiment planning and execution, main text and figure generation, and in-depth reviewing. The author reviewed all AI-assisted work and is responsible for the content.

A. Departures from ACDC’s Protocol

In this paper’s setup the agent is given the camera pose and floor height, whereas ACDC, by its published description, is given the intrinsics only and fits the floor plane from monocular depth; the asset catalog here has no thumbnails, whereas ACDC matches against rendered asset snapshots. ACDC does not state how predictions are paired with ground truth, and its sim-to-sim evaluation code was never released. The pairing rule used here, one Hungarian assignment on box centres as described in Appendix C, is therefore this paper’s own choice rather than ACDC’s. ACDC reports its orientation error only over objects outside a symmetry class, but its captions name that class without defining which objects fall in it. The protocol here excludes a fixed list of rotationally ambiguous objects instead, written down per object before any run, and reports the orientation error over every other object. Axis-aligned boxes are used, as ACDC’s supplementary tables specify for Bbox IoU.

B. Script-Writing Ablation

One sentence of the prompt in Section 3, placed after the description of the render script, changes what the agent builds: “Modify it or write your own scripts as needed: walls, floor, lighting and anything else in the background are yours to build; furniture must come from the catalog.” It lets the agent edit or replace the render script and build the background by any means, and the background is rendered but never scored. The sentence was absent from the prompt file for part of the campaign, so rooms E, F, G, I and J each have one Opus 5 run without it and one with the full prompt, under the same harness and reviewer instruction. Every run in Table 3 used the full prompt. The runs without it appear only here, as the (–) rows of Table 1.

Behaviour. Without the sentence no run edited the render script, and every run built its background from catalog wall, floor and ceiling assets placed in the scene file. With the sentence three runs edited the render script and one replaced it, and two rooms built their background from primitive boxes and lights instead.

Reading. Across the five rooms the mean centre error fell from 107 to 32 cm, mean Cat. fell from 0.60 to 0.49, mean Mod. was unchanged and cost rose about 1.6×. The centre error fell in E, F and G and rose in I and J. Each cell is one run, so room-level differences are indicative only. The G run with the sentence ended on a server-side API error and is scored on its final scene file.

Table 1. Opus 5 on five rooms with the prompt lacking (–) or containing (+) the script-writing sentence, one run per cell. Cat., Mod.: fraction of the N ground-truth objects with correct category / asset model; ${\mathcal{L}}_2$: mean centre error of matched objects; Ori.: mean rotation error of matched non-symmetric objects, with flips (within 2° of 90°/180°) over scored orientations; Bbox, Cen.: 3D box IoU before and after centre alignment; $N_{\mathrm{p}}$: scored objects placed (background excluded); USD: session cost; min: wall time. Mean rows: room means, ${\mathcal{L}}_2$ median in parentheses, flips and $N_{\mathrm{p}}/N$ pooled.

Room	Prompt	Cat.	Mod.	${\mathcal{L}}_2$ cm	Ori. rad	Bbox	Cen.	${\mathit{N}}_{\mathbf{p}}/\mathit{N}$	USD	min
E bath	–	9/11	9/11	33	2.79 (8/9)	0.03	0.77	9/11	73	183
	+	5/11	5/11	8	3.14 (5/5)	0.27	0.39	5/11	59	121
F dining	–	2/6	0/6	403	1.58 (3/6)	0.04	0.13	7/6	65	107
	+	1/6	1/6	11	0.00 (0/1)	0.13	0.16	1/6	185	169
G office	–	5/8	4/8	25	0.55 (1/7)	0.60	0.69	10/8	186	127
	+	6/8	6/8	10	0.08 (0/7)	0.63	0.77	9/8	252	136
I bar	–	6/14	5/14	59	0.67 (2/7)	0.27	0.59	17/14	43	84
	+	4/14	4/14	108	1.35 (3/7)	0.10	0.46	15/14	117	156
J bath	–	7/9	7/9	17	0.00 (0/8)	0.58	0.76	8/9	64	228
	+	7/9	7/9	21	0.00 (0/9)	0.65	0.83	16/9	60	130
Mean	–	0.60	0.49	107 (33)	1.12 (14/37)	0.30	0.59	51/48	86	146
	+	0.49	0.49	32 (11)	0.91 (8/29)	0.36	0.52	46/48	135	142

Table 1. Opus 5 on five rooms with the prompt lacking (–) or containing (+) the script-writing sentence, one run per cell. Cat., Mod.: fraction of the N ground-truth objects with correct category / asset model; ${\mathcal{L}}_2$: mean centre error of matched objects; Ori.: mean rotation error of matched non-symmetric objects, with flips (within 2° of 90°/180°) over scored orientations; Bbox, Cen.: 3D box IoU before and after centre alignment; $N_{\mathrm{p}}$: scored objects placed (background excluded); USD: session cost; min: wall time. Mean rows: room means, ${\mathcal{L}}_2$ median in parentheses, flips and $N_{\mathrm{p}}/N$ pooled.

Room	Prompt	Cat.	Mod.	${\mathcal{L}}_2$ cm	Ori. rad	Bbox	Cen.	${\mathit{N}}_{\mathbf{p}}/\mathit{N}$	USD	min
E bath	–	9/11	9/11	33	2.79 (8/9)	0.03	0.77	9/11	73	183
	+	5/11	5/11	8	3.14 (5/5)	0.27	0.39	5/11	59	121
F dining	–	2/6	0/6	403	1.58 (3/6)	0.04	0.13	7/6	65	107
	+	1/6	1/6	11	0.00 (0/1)	0.13	0.16	1/6	185	169
G office	–	5/8	4/8	25	0.55 (1/7)	0.60	0.69	10/8	186	127
	+	6/8	6/8	10	0.08 (0/7)	0.63	0.77	9/8	252	136
I bar	–	6/14	5/14	59	0.67 (2/7)	0.27	0.59	17/14	43	84
	+	4/14	4/14	108	1.35 (3/7)	0.10	0.46	15/14	117	156
J bath	–	7/9	7/9	17	0.00 (0/8)	0.58	0.76	8/9	64	228
	+	7/9	7/9	21	0.00 (0/9)	0.65	0.83	16/9	60	130
Mean	–	0.60	0.49	107 (33)	1.12 (14/37)	0.30	0.59	51/48	86	146
	+	0.49	0.49	32 (11)	0.91 (8/29)	0.36	0.52	46/48	135	142

C. Room-Experiment Details

Rooms. Table 2 lists the eight evaluation rooms by their obscure ids. Rooms are chosen by a fixed rule rather than manually: a list of desired room types (living room, bedroom, kitchen, bathroom, dining room, office, bar) is fixed, and at most one room per B1K house is taken. In each room only the eight largest assets are kept, with every instance of a kept model counting as a separate object; a model used in one room is excluded from all others. A room is accepted if it has at least five assets kept, at least five kept objects visible from the camera, and a scale (ACDC’s definition: the largest distance between two objects) inside ACDC’s range. All other furniture is deleted before the photograph is rendered, so the image shows exactly the scored objects and the background. ACDC does not describe how its rooms were prepared; only large furniture is kept here, so that object counts fall in the range of its table and every scored object is resolvable in one photograph. The photograph is rendered in OmniGibson from the footprint corner that sees the most kept objects, at a fixed inset and height, aimed at their centroid, with no retakes. Ground truth is every kept object visible in the photograph. Two candidate rooms were replaced before any run (one failed the recitation probe, one had too few scored objects), and no spare remains.

Scoring. Predictions are paired with ground truth by one category-agnostic Hungarian assignment on box centres with no distance gate. Unmatched ground truth counts as wrong for Cat. and Mod., scores zero IoU and is left out of ${\mathcal{L}}_2$ and Ori.; extra predictions are not penalised, so the predicted count is reported beside the scored count. The background (walls, floors, windows, doors, carpets) is rendered but never scored: predicted objects in those categories are dropped before pairing. Ori. skips a per-object list of rotationally ambiguous objects written before any run (Appendix A) and is reported raw with the count of exact quarter- or half-turn flips (fixed tolerance).

Table 2. The eight evaluation rooms.

Room	Type	Scored objects	Scale (m)
B	bedroom	8	5.3
C	commercial kitchen	9	6.1
D	living room	10	4.3
E	bathroom	11	4.9
F	restaurant dining room	6	9.0
G	private office	8	3.5
I	bar	14	6.1
J	station bathroom	9	6.1

Table 2. The eight evaluation rooms.

Room	Type	Scored objects	Scale (m)
B	bedroom	8	5.3
C	commercial kitchen	9	6.1
D	living room	10	4.3
E	bathroom	11	4.9
F	restaurant dining room	6	9.0
G	private office	8	3.5
I	bar	14	6.1
J	station bathroom	9	6.1

Figure 6. All eight rooms: input photograph and each agent’s final render from its first run, from the same fixed camera.

Figure 6. All eight rooms: input photograph and each agent’s final render from its first run, from the same fixed camera.

Claude arms. The four Claude agents ran through the Claude Agent SDK over the shipped Claude Code binary with its stock tool suite (file read, write and edit, shell, search, and sub-agent spawning). Permission prompts were bypassed, the two web tools were disabled so a model cannot look up B1K scene files online, and no MCP servers, hooks, or tools of any kind were added. One SDK default matters for interpretation: when no system prompt is supplied, the SDK launches the binary with an empty system prompt—so the Claude arms ran with Claude Code’s tools but no system prompt at all, rather than under Claude Code’s default prompt. When an agent created a reviewer, it wrote the brief and chose the reviewer’s model itself. Of the 64 Claude runs, 7 spawned no reviewer (Fable 5 in rooms B and C and in both runs of room E; Opus 4.6 in one run of room E; Fable 5.1 in one run each of rooms D and J): each launched a background render, ended its turn while waiting for it, and was scored on the scene file at that point.

Fable 5.1 arm. Claude Fable 5.1 (claude-fable-5-1, released 2026-09-01) was added after the other arms under the same settings, prompt and harness code; its two runs per room are r7 and r17, with every r17 run started after the last r7 run had finished. This arm ran on claude-agent-sdk 0.2.151 with Claude Code CLI 2.1.258, because the 0.2.144 / 2.1.239 pair the other Claude arms ran on rejects the model; the harness code is identical. The recitation probe, rerun for this model, lists one of the scored assets of room D (a sofa, 1 of 10) and one of room G (a bookcase, 1 of 8) among mostly wrong guesses, and none in the other six rooms; both assets were also placed by agents whose probe scores zero on those rooms, so the probe does not separate recall from retrieval there, and both rooms are kept. One run (room J, r7) read the first 80 lines of the benchmark’s README, a file outside the lock that describes the protocol and file layout and holds no room, object or camera data; the harness audit flags the run for a file name that text contains, the run is kept and pooled, and without it the arm’s column in Table 3 would read 0.57 / 0.46 / 60 (40) / 0.94 / 0.25 / 0.51 for Cat., Mod., ${\mathcal{L}}_2$, Ori., Bbox IoU and Cen. IoU. Three runs (rooms E and F, r7; room G, r17) listed path names under an unlocked pre-migration copy of earlier run records while locating their workspace with find; no tool input read a file there. The first r7 attempts of rooms C and G ended after 12 to 14 minutes with no render, each waiting on a background render, and were rerun by decision of the author; the first attempts are retained beside the run records but not reported.

Codex arms. GPT-6 Astra, GPT-5.6 Sol and GPT-5.5 ran through OpenAI Codex in non-interactive mode with approvals and sandboxing bypassed—the analogue of the Claude arms’ permission bypass, needed to launch the simulator—and with web search, browser, computer use, image generation, apps, and plugins disabled, ignoring any user configuration. Codex offers no empty-prompt option, so these arms ran under its vendor system prompt plus a per-turn multi-agent policy message. Its tool surface is a single shell tool with patch-apply and image-view commands rather than Claude Code’s separate file tools, and its reviewer starts from a copy of the agent’s prompt messages but none of its turns, followed by the agent’s brief, where Claude Code’s sub-agent receives the brief alone; its policy discourages sub-agents unless the user asks for them, which the prompt’s “until reviewer agent approves” clause satisfied in every room. Reasoning effort was raised to high for all three models; the Claude arms ran at their default reasoning settings.

GPT-6 Astra arm. GPT-6 Astra (gpt-6-astra, released 2026-09-03) was added after the other arms under the same settings, prompt and harness code apart from the model’s list-price entry; its two runs per room are r8 and r18, with every r18 run started after the last r8 run had finished. This arm ran on Codex CLI 0.153.4 where the other Codex arms ran on 0.150.1. The recitation probe was not run for the other Codex arms. For this model it was run through Codex with every tool disabled, since the probe script issues its question through the Claude Agent SDK and this setup reaches OpenAI models only through that CLI, and scores zero on every reported room: on seven rooms the model lists no asset, and on room J it lists six pairs none of whose ids is a catalog model. Two runs (room J, r8; room D, r18) are flagged by the harness audit because the reviewer’s approval note, written into the workspace with Codex’s patch tool, ends with a sentence stating that no file under the locked scene directory was read; the harness exempts the prompt’s rule as the agent relays it to a reviewer but not this past-tense statement in a written file, so the flag stood. The hit is the class the audit of the recorded Codex arms also raised on three GPT-5.5 records, where the audit words appear in the rule relayed to a reviewer, in listing commands that exclude the directory and in one workspace edit of render.py, none a read of a locked file; the campaign re-audit found no ground-truth access and kept those records with their labels. No shell command in the arm names that directory except as a negated filter in rg file listings, the opening listing of 13 of the 16 runs and four later listings in two of them, a form the audit already exempts, and no ground-truth content was returned; both runs are kept and pooled, and without them the arm’s column in Table 3 would read 0.71 / 0.64 / 27 (17) / 0.46 / 0.33 / 0.61 for Cat., Mod., ${\mathcal{L}}_2$, Ori., Bbox IoU and Cen. IoU, with 107 of 131 objects placed over the 14 runs. Two runs (room F, r8; room J, r18) met the lock while listing files with rg: the permission-denial lines named the locked directories, 24 in the first run and one in the second, and returned no entry or content from inside any of them; for this arm the lock also covered the main checkout’s run records and the pre-migration copies of earlier run records, and no command named them. No run was rerun, none resumed after a rate limit, and every run spawned exactly one reviewer. Both runs of room D are marked by the harness for stream reconnections during the session; each turn completed with exit code 0 and a scored scene file, as on five records of the other Codex arms. Cost is the same list-price estimate as for the other Codex arms, at $10, $1 and $50 per million input, cached-input and output tokens, since the runs were billed to a subscription.

Limits and the one capped run. No turn, wall-clock, or cost cap was imposed on any reported run except one: the earlier Opus 5 run in room C was started under an earlier 400-turn / 90-minute / $30 cap and finished within all three, so it is kept. Two per-tool limits applied throughout: a single shell command times out after 10 minutes, and one SDK message may carry at most 64 MB. On an API rate-limit rejection the harness sleeps until the limit window resets and resumes the same session with “Continue where you left off.”; two reported runs (Fable 5, room G; Fable 5.1, room I) took that path, and the wait is excluded from wall time.

Table 3. Room reconstruction on ACDC’s protocol: ACDC quoted on its own rooms; seven agents under one prompt on this paper’s rooms

	ACDC	Claude Agent SDK				Codex
	(quoted)	Opus 5	Fable 5.1	Fable 5	Opus 4.6	GPT-6 Astra	GPT-5.6 Sol	GPT-5.5
rooms / scored objects	4/39	8/75	8/75	8/75	8/75	8/75	8/75	8/75
objects placed / in the rooms	–	170/150	135/150	143/150	147/150	123/150	110/150	282/150
Cat. ↑	1.00	0.58	0.55	0.48	0.25	0.72	0.46	0.27
Mod. ↑	1.00	0.52	0.44	0.31	0.06	0.66	0.32	0.11
${\mathcal{L}}_2$ cm ↓, mean (median)	8 (6)	54 (39)	59 (46)	75 (54)	147 (153)	24 (14)	101 (76)	123 (120)
Ori. rad ↓	0.08†	0.94‡	0.93‡	1.01‡	1.22‡	0.48‡	1.05‡	1.44‡
flips / scored	–	39/108	36/105	32/101	36/106	19/96	30/89	25/95
Bbox IoU ↑	0.66	0.40	0.24	0.14	0.04	0.36	0.09	0.02
Cen. IoU ↑	0.74	0.55	0.50	0.39	0.24	0.62	0.30	0.17
cost / room ($), median	–	120	13	26	7	9§	6§	9§
wall / room (min), median	–¶	132	28	43	38	17	26	23

Cat. and Mod.: fraction of ground-truth objects with the correct category and asset model; ${\mathcal{L}}_2$: distance between matched box centres (cm); Ori.: rotation error (rad) over objects not excluded as symmetric; Bbox IoU: 3D bounding-box IoU; Cen. IoU: IoU after centre alignment. Agents may place more scored-category objects than a room contains; extra predictions are unpenalised, and the objects-placed row exposes them. ACDC’s column is its published sim-to-sim table, averaged here over its four rooms (one run each); the agent columns are this paper’s rooms under the same metric definitions, two runs per room, both pooled. Metric rows are means over rooms (${\mathcal{L}}_2$ also the median run); cost and wall time are the median run. One Opus 5 room finished within an earlier cap. One Fable 5.1 run is kept with a harness audit flag for reading the benchmark’s README, a file outside the lock with no room data (Appendix C). Two GPT-6 Astra runs are kept with a harness audit flag for a reviewer note that names the locked directory to state it was not read (Appendix C). Departures from ACDC’s published description are listed in Appendix A. †after ACDC’s symmetry restriction, which its captions name but do not define. ‡raw; the next row counts exact quarter- or half-turn flips (fixed tolerance). §list-price estimate (runs billed to a subscription). ¶ACDC publishes per-object timings, no per-room time.

Table 3. Room reconstruction on ACDC’s protocol: ACDC quoted on its own rooms; seven agents under one prompt on this paper’s rooms

	ACDC	Claude Agent SDK				Codex
	(quoted)	Opus 5	Fable 5.1	Fable 5	Opus 4.6	GPT-6 Astra	GPT-5.6 Sol	GPT-5.5
rooms / scored objects	4/39	8/75	8/75	8/75	8/75	8/75	8/75	8/75
objects placed / in the rooms	–	170/150	135/150	143/150	147/150	123/150	110/150	282/150
Cat. ↑	1.00	0.58	0.55	0.48	0.25	0.72	0.46	0.27
Mod. ↑	1.00	0.52	0.44	0.31	0.06	0.66	0.32	0.11
${\mathcal{L}}_2$ cm ↓, mean (median)	8 (6)	54 (39)	59 (46)	75 (54)	147 (153)	24 (14)	101 (76)	123 (120)
Ori. rad ↓	0.08†	0.94‡	0.93‡	1.01‡	1.22‡	0.48‡	1.05‡	1.44‡
flips / scored	–	39/108	36/105	32/101	36/106	19/96	30/89	25/95
Bbox IoU ↑	0.66	0.40	0.24	0.14	0.04	0.36	0.09	0.02
Cen. IoU ↑	0.74	0.55	0.50	0.39	0.24	0.62	0.30	0.17
cost / room ($), median	–	120	13	26	7	9§	6§	9§
wall / room (min), median	–¶	132	28	43	38	17	26	23

Cat. and Mod.: fraction of ground-truth objects with the correct category and asset model; ${\mathcal{L}}_2$: distance between matched box centres (cm); Ori.: rotation error (rad) over objects not excluded as symmetric; Bbox IoU: 3D bounding-box IoU; Cen. IoU: IoU after centre alignment. Agents may place more scored-category objects than a room contains; extra predictions are unpenalised, and the objects-placed row exposes them. ACDC’s column is its published sim-to-sim table, averaged here over its four rooms (one run each); the agent columns are this paper’s rooms under the same metric definitions, two runs per room, both pooled. Metric rows are means over rooms (${\mathcal{L}}_2$ also the median run); cost and wall time are the median run. One Opus 5 room finished within an earlier cap. One Fable 5.1 run is kept with a harness audit flag for reading the benchmark’s README, a file outside the lock with no room data (Appendix C). Two GPT-6 Astra runs are kept with a harness audit flag for a reviewer note that names the locked directory to state it was not read (Appendix C). Departures from ACDC’s published description are listed in Appendix A. †after ACDC’s symmetry restriction, which its captions name but do not define. ‡raw; the next row counts exact quarter- or half-turn flips (fixed tolerance). §list-price estimate (runs billed to a subscription). ¶ACDC publishes per-object timings, no per-room time.

D. Farm Reconstruction Traces

This appendix summarises working traces from the first eight Opus 5 runs, showing how each of those eight twins’ camera, plants and man-made structures were derived from the photograph. Trace excerpts are quoted from the run logs with symbols typeset and wording unchanged.

Campaign prompt. The first eight Opus 5 twins and the 46 GPT-6 Astra twins were set by the following prompt to the top-level agent, with local paths replaced by placeholders (the Astra campaign used the same prompt with the line “Use GPT-6 Astra as subagents with high effort.”):

Goal: For each image under input scene folder, iterate on scripts to recreate the digital twin in IsaacSim and render.

How: Spawn a workflow. Generate agent re-generate and re-render until Reviewer agent approves for general similarity between input scene image and render image. Use Opus 5 as subagents with medium effort.

Instructions:

- Input scene folder: <input folder>

- Output reconstructed scene folders:

     - Scripts: <scripts folder>

     - Render: <renders folder>

- Store temporary artifacts under: <artifacts folder>

- Reference example scene twin generation:

     - Script: <reference script>

     - Image: <reference image>

     - Reconstructed render in IsaacSim: <reference render>

     - Real-time tracing during iterations; Path tracing for the final render

     - Cap at 3 reviewer rounds

     - Stop whenever similarity is reached within existing assets and capability, or when new round cannot yield meaningful improvement.

Camera estimation. Each farm run receives no camera information, so the agent recovers the viewpoint from the photograph before it can place any object. Table 4 lists the cue each run recorded in its scene script together with the camera it solved, taken from the Opus 5 run that produced that campaign’s final render. The solved viewpoints span eye heights of 1.55 to 2.30 m, downward pitches of 2.3 to 46.8 degrees, lens fields of view of 40 to 66 degrees, and both landscape and portrait framings, which covers ground-level alley views, oblique field views, a steep look-down over beds, and a greenhouse interior.

The initial estimate comes from explicit projective fits rather than guesswork. In the greenhouse strawberry run the agent wrote fitting three readable bag rows (image $y\approx 1150/590/490$) to $y=y_h+f\Delta z/d$ under a constant bed pitch solves to horizon $y_h\approx$ 247–320 px (0.19–0.25 H); with $f=1663$ px (hfov 60 deg on 1920 px) that gives camera pitch ≈ 10.9 deg down, then adopted an eye height of 1.67 m. In 6 of the 8 runs the camera stayed fixed from the first round onward and the remaining rounds went to the scene itself.

Two runs corrected the viewpoint in later rounds in response to the reviewer critique. The field strawberry run held eye height, pitch and lens fixed while rotating its yaw off the row axis from 17 to 21.5 and then 30 degrees, after the reviewer measured the near bed rising more steeply across the photograph than in the render. The greenhouse strawberry run raised the camera itself, from 1.67 to 1.72 and finally 2.30 m (Figure 5), and its round-three log states the geometric reason, the near-pair ratio $v_0/v_1=773/593$ forces bed pitch $\approx 0.30d_0$, and the condition to see any floor over a bag crown ($\mathit{eye}/(\mathit{eye}-\mathit{bag}\_\mathit{top})·d_0<d_0+\mathit{pitch}$) then only holds from a viewpoint well above the bags.

Plant configuration. The runs place plants pre-grown with the botanical model and fit their scale, spacing, density and colour to the photograph. The first apple run back-projected its most prominent tree, base at 0.665 H → 7.2 m away, top at 0.305 H →≈2.5 m tall, subtending ≈2.4 m wide → cachedapple_bush_*.usdc(3.7 m native) scaled 0.62–0.74. The high-angle lettuce run reported head sizes measured by unprojection: ≈0.30 m near, ≈0.14 m far and scaled its rows by depth accordingly, so plant scale falls with depth as the photograph shows. Colour is fitted by measurement rather than by eye. The coastal broccoli run sampled matched patches of photograph and render, crop-near (53,63,46) vs (73,87,48) → render too bright and blue-starved (B/G 0.55 vs 0.73), and retuned its leaf tint against a fresh sample in each subsequent round.

Primitive structures. Man-made structures are composed from colored box, cylinder and mesh primitives at dimensions back-projected from the photograph. The greenhouse run framed its roof as 0.11 m posts on a 2.73 × 4.1 m grid […] all dark weathered timber/steel so it reads as the photo’s dark tangle against blown-out white film, and carried a 0.13 m irrigation main across the frame at the image height the photograph shows. The fleece-field broccoli run laid two crumpled fleece ribbons ≈0.5 m wide along its tramlines and closed the horizon with a white farmhouse with red gable roof at the far edge of the field. The coastal broccoli run strung a telegraph line with sagging wires over poles receding toward the horizon, and the first apple run placed a small bin at the image position the photograph shows.

E. Farm Validation Metric

Twin fidelity is quantified with two independent pretrained vision encoders, DINOv2 (Oquab et al. 2024) and CLIP (Radford et al. 2021). Each final render and each reference photograph is embedded once per encoder, using the DINOv2 base model and the CLIP ViT-B/32 image tower at pinned checkpoint revisions, and every embedding is L2-normalised. The similarity of a render to a photograph is the cosine between their embeddings. Each twin’s render is scored against every reference photograph in the pool, and the rank of the twin’s own photograph is recorded, so rank one means the render is closer to its own photograph than to any other reference.

Two calibrations put the cosine values on a natural scale. First, the mean similarity of matched render–photograph pairs is compared with the mean over all mismatched pairs. Second, the reference photographs of the same crop are scored against each other, and the mean over those pairs is the level that real images of the same crop type reach, so a matched-pair mean at or above that level indicates the renders sit at the similarity level of real same-crop photographs. Both encoders are reported because they respond to different cues, and where they disagree the disagreement is reported rather than resolved. Table 5 lists the values for each agent’s 46 twins of Section 5, computed on the final path-traced renders, which the Opus 5 campaign produced at 2600 px wide and 2048 samples per pixel and the Astra campaign at 2000 px and 1024 samples per pixel. Under DINOv2 24 of the 46 Astra twins and 22 of the 46 Opus 5 twins rank first, with median ranks of one and two, matched pairs average 0.52 for Astra and 0.57 for Opus 5 against 0.20 for mismatched pairs, and the 87 same-crop photograph pairs average 0.49 with a cross-crop photograph mean of 0.20. Under CLIP 22 of the 46 Astra twins and 25 of the 46 Opus 5 twins rank first, with median ranks of two and one, matched pairs average 0.76 for Astra and 0.80 for Opus 5, against 0.67 and 0.69 in the same order, and same-crop photograph pairs average 0.79 with a cross-crop photograph mean of 0.71, so CLIP separates matched from mismatched pairs by far less than DINOv2 does.

Table 5. Render-to-photograph similarity for the 46 farm twins of each agent, Codex with GPT-6 Astra and Claude Code with Opus 5. Cosine is between the twin’s final render and its own photograph; rank is the position of the own photograph among the 46 reference photographs (1 = closest).

	DINOv2				CLIP
	Astra		Opus 5		Astra		Opus 5
Twin	cosine	rank	cosine	rank	cosine	rank	cosine	rank
apple 1	0.63	1	0.72	1	0.70	4	0.74	3
apple 2	0.68	1	0.60	1	0.82	1	0.79	1
broccoli 1	0.53	3	0.65	1	0.78	7	0.76	4
broccoli 2	0.34	10	0.52	1	0.71	3	0.84	4
lettuce 1	0.76	1	0.86	1	0.82	1	0.88	4
lettuce 2	0.58	3	0.68	2	0.74	1	0.80	1
strawberry 1	0.64	1	0.47	9	0.88	1	0.86	1
strawberry 2	0.47	2	0.49	3	0.85	2	0.79	5
apple 3	0.46	1	0.52	1	0.67	27	0.72	13
apple 4	0.40	1	0.53	1	0.65	4	0.73	5
apple 5	0.40	1	0.49	1	0.77	1	0.73	1
strawberry 3	0.71	1	0.73	1	0.81	2	0.84	2
strawberry 4	0.76	1	0.87	1	0.89	1	0.85	1
strawberry 5	0.49	2	0.69	1	0.73	1	0.86	1
lettuce 3	0.75	1	0.64	1	0.83	2	0.77	2
lettuce 4	0.63	2	0.63	2	0.86	1	0.77	1
lettuce 5	0.62	3	0.66	3	0.91	1	0.92	1
blueberry 1	0.38	9	0.56	3	0.75	12	0.79	2
blueberry 2	0.49	1	0.65	2	0.74	1	0.87	1
blueberry 3	0.55	1	0.56	2	0.72	10	0.75	5
blueberry 4	0.54	1	0.42	2	0.74	9	0.80	2
table grape 1	0.68	1	0.66	1	0.73	4	0.90	1
table grape 2	0.42	5	0.45	3	0.58	26	0.72	6
table grape 3	0.82	1	0.76	1	0.76	3	0.79	2
table grape 4	0.72	1	0.68	1	0.69	13	0.72	1
table grape 5	0.53	4	0.46	5	0.72	1	0.79	1
peach 1	0.35	12	0.35	9	0.72	18	0.64	20
peach 2	0.38	8	0.54	2	0.79	1	0.82	1
peach 3	0.28	10	0.51	2	0.76	1	0.84	2
peach 4	0.38	5	0.55	1	0.73	1	0.87	1
peach 5	0.14	18	0.36	4	0.67	7	0.70	11
sweet cherry 1	0.56	1	0.65	2	0.69	4	0.70	5
sweet cherry 2	0.46	5	0.56	3	0.88	1	0.86	1
sweet cherry 3	0.72	1	0.72	2	0.86	1	0.88	1
sweet cherry 4	0.65	3	0.74	1	0.85	1	0.88	1
sweet cherry 5	0.71	1	0.72	3	0.83	1	0.82	1
tomato 1	0.49	1	0.52	2	0.67	12	0.73	34
tomato 2	0.55	1	0.53	1	0.88	1	0.73	1
tomato 3	0.64	1	0.71	1	0.76	2	0.80	1
tomato 4	0.36	5	0.35	2	0.81	1	0.83	1
tomato 5	0.57	1	0.66	1	0.91	1	0.89	1
sweet potato 1	0.59	2	0.50	1	0.76	16	0.85	5
sweet potato 2	0.16	24	0.26	4	0.68	1	0.84	1
sweet potato 3	0.16	21	0.14	6	0.67	5	0.79	1
sweet potato 4	0.37	1	0.54	2	0.76	9	0.77	1
sweet potato 5	0.55	2	0.59	1	0.67	25	0.72	3
matched mean	0.52		0.57		0.76		0.80
mismatched mean	0.20		0.20		0.67		0.69
same-crop photograph pair mean	0.49				0.79

Table 5. Render-to-photograph similarity for the 46 farm twins of each agent, Codex with GPT-6 Astra and Claude Code with Opus 5. Cosine is between the twin’s final render and its own photograph; rank is the position of the own photograph among the 46 reference photographs (1 = closest).

	DINOv2				CLIP
	Astra		Opus 5		Astra		Opus 5
Twin	cosine	rank	cosine	rank	cosine	rank	cosine	rank
apple 1	0.63	1	0.72	1	0.70	4	0.74	3
apple 2	0.68	1	0.60	1	0.82	1	0.79	1
broccoli 1	0.53	3	0.65	1	0.78	7	0.76	4
broccoli 2	0.34	10	0.52	1	0.71	3	0.84	4
lettuce 1	0.76	1	0.86	1	0.82	1	0.88	4
lettuce 2	0.58	3	0.68	2	0.74	1	0.80	1
strawberry 1	0.64	1	0.47	9	0.88	1	0.86	1
strawberry 2	0.47	2	0.49	3	0.85	2	0.79	5
apple 3	0.46	1	0.52	1	0.67	27	0.72	13
apple 4	0.40	1	0.53	1	0.65	4	0.73	5
apple 5	0.40	1	0.49	1	0.77	1	0.73	1
strawberry 3	0.71	1	0.73	1	0.81	2	0.84	2
strawberry 4	0.76	1	0.87	1	0.89	1	0.85	1
strawberry 5	0.49	2	0.69	1	0.73	1	0.86	1
lettuce 3	0.75	1	0.64	1	0.83	2	0.77	2
lettuce 4	0.63	2	0.63	2	0.86	1	0.77	1
lettuce 5	0.62	3	0.66	3	0.91	1	0.92	1
blueberry 1	0.38	9	0.56	3	0.75	12	0.79	2
blueberry 2	0.49	1	0.65	2	0.74	1	0.87	1
blueberry 3	0.55	1	0.56	2	0.72	10	0.75	5
blueberry 4	0.54	1	0.42	2	0.74	9	0.80	2
table grape 1	0.68	1	0.66	1	0.73	4	0.90	1
table grape 2	0.42	5	0.45	3	0.58	26	0.72	6
table grape 3	0.82	1	0.76	1	0.76	3	0.79	2
table grape 4	0.72	1	0.68	1	0.69	13	0.72	1
table grape 5	0.53	4	0.46	5	0.72	1	0.79	1
peach 1	0.35	12	0.35	9	0.72	18	0.64	20
peach 2	0.38	8	0.54	2	0.79	1	0.82	1
peach 3	0.28	10	0.51	2	0.76	1	0.84	2
peach 4	0.38	5	0.55	1	0.73	1	0.87	1
peach 5	0.14	18	0.36	4	0.67	7	0.70	11
sweet cherry 1	0.56	1	0.65	2	0.69	4	0.70	5
sweet cherry 2	0.46	5	0.56	3	0.88	1	0.86	1
sweet cherry 3	0.72	1	0.72	2	0.86	1	0.88	1
sweet cherry 4	0.65	3	0.74	1	0.85	1	0.88	1
sweet cherry 5	0.71	1	0.72	3	0.83	1	0.82	1
tomato 1	0.49	1	0.52	2	0.67	12	0.73	34
tomato 2	0.55	1	0.53	1	0.88	1	0.73	1
tomato 3	0.64	1	0.71	1	0.76	2	0.80	1
tomato 4	0.36	5	0.35	2	0.81	1	0.83	1
tomato 5	0.57	1	0.66	1	0.91	1	0.89	1
sweet potato 1	0.59	2	0.50	1	0.76	16	0.85	5
sweet potato 2	0.16	24	0.26	4	0.68	1	0.84	1
sweet potato 3	0.16	21	0.14	6	0.67	5	0.79	1
sweet potato 4	0.37	1	0.54	2	0.76	9	0.77	1
sweet potato 5	0.55	2	0.59	1	0.67	25	0.72	3
matched mean	0.52		0.57		0.76		0.80
mismatched mean	0.20		0.20		0.67		0.69
same-crop photograph pair mean	0.49				0.79

F. Photograph Credits

The 46 reference photographs are reproduced under the Creative Commons or public-domain terms stated on their source pages, and Table 6 lists the license and source page of each and the author where the source page names one.

Table 6. Credits for the 46 reference photographs. Titles are the Wikimedia Commons file names for the photographs taken from Commons, and the source column links to the page that states the license.

Twin	Title	Author	License	Source

Table 6. Credits for the 46 reference photographs. Titles are the Wikimedia Commons file names for the photographs taken from Commons, and the source column links to the page that states the license.

Twin	Title	Author	License	Source

G. Tool Usage of the Reconstruction Agent

Table 7 counts every tool call in the transcripts of the 16 Opus 5 room runs of Section 3, the two runs per room that Table 3 pools, main agent and sub-agents pooled and split by round: round 1 is the earlier of the two runs of each room and round 2 the later, started after every round-1 run had finished, under the same prompt and model. Definitions are under the table; Section 4 interprets the distribution.