Preprint
Review

This version is not peer-reviewed.

Visual Generative AI for Extended Reality: A Survey

Submitted:

25 October 2026

Posted:

27 October 2026

You are already at the latest version

Abstract
Visual generative models create objects, spatial scenes, time-varying 3D content, and action-conditioned observations for extended reality (XR). Using these outputs requires knowing which viewing, editing, and state-changing operations they support, and which system components supply those operations. This critical narrative review catalogues 204 methods and systems and traces generated outputs into specific XR operations. Four comparison axes organize content units, generative mechanisms, representations, and operating stages; common mathematical definitions clarify conditioning and update rules. An evidence map records eight capability dimensions for a purposively selected subset of 31 methods and systems. The synthesis combines these profiles with source-specific quantitative comparisons and deployment studies, identifying the components, configurations, and endpoints supporting capability claims. Building on paradigm-level comparisons, this analysis distinguishes the evidence needed to generate editable structure, preserve an accepted modification, and respond to tracked input. Visual recall and modification retention require different references, while generation throughput and input-to-display latency describe different parts of the response path. Worked task comparisons explain how these distinctions change the choice between prepared geometry with local rendering and continuing observation generation. We derive task-specific selection criteria and propose matched evaluations of binocular viewing, preview replacement, persistent changes, response scheduling, and physical-space alignment. The resulting framework connects model selection to explicit task requirements and identifies the additional tests needed to assess XR use. The survey homepage is available at https://edhuang1.github.io/xr-survey/.
Keywords: 
;  ;  ;  ;  ;  

1. Introduction

Visual generative models contribute geometry, appearance, layout, motion, and continuing observations to extended reality (XR). In virtual reality (VR), augmented reality (AR), and mixed reality (MR), users also expect this content to respond to navigation and manipulation. A generated chair may look convincing in one view, yet placing it, walking around it, and moving its parts require different structures and controls. This review asks which generated outputs and supporting components enable the operations an XR application needs.
The answer depends on how content is constructed and what remains available during use. DreamFusion optimizes 3D content through an image-generation prior, whereas TRELLIS samples structured 3D latents [229,344]. At scene scale, Text2Room extends a textured mesh through generated views and geometric fusion; Holodeck assembles retrieved assets using generated spatial constraints [103,365].
Figure 1. Review map linking content families, mechanisms, data and evaluation targets, and XR operations. The asset mesh is schematic. Images from Bahmani et al.(2024)Bahmani, Skorokhodov, Rong, Wetzstein, Guibas, Wonka, Tulyakov, Park, Tagliasacchi, and Lindell [9], Fridman et al.(2023)Fridman, Abecasis, Kasten, and Dekel [71], Sun et al.(2026a)Sun, Zhang, Wang, Wu, Wang, Wang, Wang, Zhang, Wang, and Guo [285] show scenes, time-varying 3D, and action-conditioned observations.
Figure 1. Review map linking content families, mechanisms, data and evaluation targets, and XR operations. The asset mesh is schematic. Images from Bahmani et al.(2024)Bahmani, Skorokhodov, Rong, Wetzstein, Guibas, Wonka, Tulyakov, Park, Tagliasacchi, and Lindell [9], Fridman et al.(2023)Fridman, Abecasis, Kasten, and Dekel [71], Sun et al.(2026a)Sun, Zhang, Wang, Wu, Wang, Wang, Wang, Zhang, Wang, and Guo [285] show scenes, time-varying 3D, and action-conditioned observations.
Preprints 240396 g001
Temporal generation introduces another distinction: 4D-fy and DreamGaussian4D prepare time-varying representations, while WorldPlay generates observations from actions and history [9,243,285]. The former retain content that can be rendered at a selected time and viewpoint; WorldPlay synthesizes new observations from its available context. This difference places rendering and generation at different points in the response path.
XR deployments reveal where these operations are introduced. CoGen3D adds headset loading and manipulation to Hunyuan3D 2.0 outputs; Dreamcrafter uses proxies while final generation runs asynchronously [132,302,395]. Evaluation must follow content through this handoff. Generation benchmarks test defined content tasks [61,193], whereas XR applications may require manipulation, paired eye views, or responses to tracked input. Generation throughput measures output rate; input-to-display latency can also include scheduling, transfer, decoding, rendering, and display.
Relation to existing surveys. Existing reviews already connect generative techniques to XR devices, workflows, and evaluation [213]. The visual GenAI/MR survey cross-codes models, interaction modalities, and evaluation; it also examines hybrid systems and reversible spatial proxies [308]. Reviews of immersive human–AI interaction and embodied 3D generation address user intent, iterative processing, simulator compatibility, physical validity, and task performance [212,372]. Complementary surveys cover VR applications, multimodal generation, and immersive-video creation, delivery, and display [109,237,290].
World-model surveys organize representations, prediction targets, and downstream tasks [150,165,258]; an action–state–observation analysis relates visual generation directly to game-engine state [167]. Most closely, Xie et al. compare four scene-generation paradigms against seven capability criteria, provide individual-method capability profiles, group quantitative results by protocol, and map application requirements to paradigms. Their analysis includes XR efficiency, persistence, and the distinction between prescribed 4D motion and interaction [347]. We build on these comparisons in the configuration-level synthesis below.
Our unit of comparison is the path from a generated output to a specified XR operation. We identify the representation exposed to the application, the component supplying the operation, and the configuration and endpoint actually tested. Evidence for generating separate parts concerns the available editing unit; evidence for manipulating imported objects concerns an implemented operation. Retention requires testing whether the accepted edit survives leaving and returning. These distinctions determine whether a reader can select a generator, needs an additional interaction component, or must still test retention. Section 5.1, Section 5.2, Section 5.3, Section 5.4, Section 5.5 and Section 5.6 connect such evidence across model and system studies; Section 6 proposes matched tests where that connection remains unmeasured. This synthesis extends capability profiling and interaction-workflow analysis through explicit evidence chains.
This critical narrative review catalogues 204 methods and systems and 37 resources, with targeted retrieval through 23 October 2026. A purposively selected subset of 31 methods and systems receives detailed capability coding. Appendix A explains selection, source inspection, and coding boundaries. Figure 2 counts catalogue papers by bibliographic year: formal publication year, or initial preprint year for preprint-only records. Each work is counted once, merging preprints with subsequent publications. The counts summarize the review catalogue.
Contributions. The review develops three connected analyses:
1.
Mechanisms and operational structure. Four axes describe content units, generative mechanisms, representations, and operating stages. Common mathematical definitions connect generated units to their conditioning inputs, retained representations, and update operators (Section 2.1 and Section 3). This shared description separates camera/time queries, observation generation, and state-changing operations.
2.
Evidence across models and deployments. Capability evidence is assessed separately from the descriptive axes. A source-linked map records evidence types across eight capability dimensions for 31 methods and systems under inspected configurations. Protocol-specific quantitative comparisons accompany these profiles. Model-to-system traces identify where delivery, interaction, and display components supply the reported operation (Figure 10; Section 4 and Section 5.1).
3.
Task-specific selection and evaluation. Worked comparisons relate the required editable unit, retained state, and response path to qualifying evidence (Section 5.6). Five proposed matched evaluation designs target transfers from model-level results to XR use (Section 6).
Reading routes. Readers entering XR can start with Section 2.2, then use the analytical framework in Section 2.1 to orient the method comparisons. Section 2.3 and Section 2.4 supply representation and learning background as needed. Readers choosing a pipeline can start with the worked tasks in Section 5.6, then follow the method, evidence-profile, and timing references.

2. Preliminaries and Review Framework

We define the comparison axes, then introduce the XR, spatial-representation, and generative-learning concepts needed to interpret them. Scene-generation and world-model surveys provide complementary introductions to representations and predictive modeling [150,165,258,347].

2.1. Analytical Framework for the Review

We compare each method along four axes: content unit, generative mechanism, representation, and operating stage. Content units organize the method families; the other axes support comparisons within and across them. A method configuration may combine several mechanisms, representations, or operating stages. Figure 3 contrasts generated observations with retained content that an application can query or update.

2.1.1. Content Units and Types of Change

Content units.An asset supplies an object or localized element; a spatial scene supplies an arrangement or connected environment. Time-varying 3D content supplies a prepared temporal representation; an action-conditioned world produces observations in response to inputs during use.
These units organize Section 3 by the output exposed to an application. A video may supervise a prepared 3D animation or serve as an observation generated during interaction. For each output, we distinguish viewpoint changes, object dynamics, action responses, and information retained for later queries.

2.1.2. Generative Mechanisms and Representations

The mechanism axis records how operators construct or change content. They may fit instance parameters to observations or a learned prior, sample a learned representation, or assemble assets from layouts. Other operators generate observations that may be fused into retained geometry. Learned models, procedural rules, and explicit constraints can participate in the same pipeline; Section 3 compares their conditions and update rules.
Representation roles.The internal representation is manipulated by the generator; the rendering representation supplies image formation; the delivered artifact is handed to another component. A shared file format does not establish that the same structure or operation survives every stage.
A point cloud used to condition generation and a mesh delivered for interaction serve different purposes, even when describing the same scene. The method and deployment comparisons therefore record conversion and system integration as explicit steps.

2.1.3. Operating Stages and System Configurations

The operating-stage axis distinguishes offline generation before viewing from runtime rendering of available content. Asynchronous editing keeps existing content or a proxy displayed, whereas streaming generation produces content as context or actions arrive. A system may combine these stages to schedule work across its components.
Figure 4 locates generation on the response paths explained in Section 2.2.3. Table 6 later compares source-reported costs at their stated endpoints.
A deployment configuration records the generator and version, asset processing, interaction components, device and compute arrangement, and task. It follows the implementation, including streams without conventional engines or exported assets.

2.1.4. XR Capabilities and Evidence Targets

The four axes describe how a method works; evidence is examined separately along eight capability dimensions. They include spatial agreement (S), binocular viewing (B), temporal consistency and persistence (T), and conditioning and control (C). The remaining dimensions are interaction (I), delivery and engine use (D), operating cost (O), and XR deployment (X). Each observation identifies a property, its reference, and the responsible component, from model output to user task. Section 4 develops the evaluation targets; Figure 10 summarizes the evidence profiles.

2.2. XR Viewing and Interaction

Coordinate frames determine which view is requested, perception determines how paired views are experienced, and the interaction loop determines when an action becomes visible.

2.2.1. XR Displays, Viewpoints, and Spatial Coordinates

Extended reality (XR) places digital information in a spatial experience. Virtual reality (VR) presents a predominantly synthetic environment; augmented reality (AR) combines digital content with a physical view registered to three-dimensional coordinates. Mixed reality (MR) combines physical and virtual elements, with usage varying across research and commercial systems. The reality–virtuality continuum and subsequent MR research document these combinations and terminology [8,204,279]. Applications can cross modes: MagicBook moves from a physical book to overlaid models and immersive scenes [18].
A head-mounted display (HMD) presents images near the eyes while the system estimates the wearer’s viewpoint. The renderer uses tracked head pose to update views, making a world-anchored object appear fixed as the viewer moves. This coupling of tracking and rendering was central to early head-mounted graphics [287]. Three degrees of freedom (3DoF) describe orientation around three axes. Six degrees of freedom (6DoF) add translation along three axes. Turning the head changes viewing direction; leaning sideways also changes surface visibility and foreground motion relative to the background. A panoramic image supports looking around from its capture center, whereas convincing translational exploration requires additional geometric or view-dependent information. Tracking technologies differ in precision, operating volume, latency, and sensitivity to occlusion or environmental conditions [329].
Spatial content passes through object (O), world (W), headset, and eye (E) coordinate frames. Let T a b map coordinates from frame b into frame a. A rigid pose and the resulting eye projection can be written as
T a b = R a b t a b 0 T 1 ∈ SE ( 3 ) , X ˜ E = T E W T W O X ˜ O , Z E u ˜ = K E X E , u ˜ = ( u 1 , u 2 , 1 ) T .
Here R a b is a rotation, satisfying R a b T R a b = I and det R a b = 1 ; t a b is a translation. A tilde denotes homogeneous coordinates, and SE ( 3 ) is the set of these rigid transformations. In the ideal pinhole projection, K E contains the eye camera’s intrinsic parameters, Z E > 0 is forward depth, and ( u 1 , u 2 ) are pixel coordinates. Each eye has its own calibrated offset from the tracked headset and its own projection parameters, jointly defining the view v. Rigid pose does not include scale: the object’s dimensions must first agree with the world’s units. A chair’s placement can then align its support surface with the floor. The same transformation chain maps tracked hands and controller rays into the world for selection and contact tests. Figure 5(a) illustrates the frames and transformation chain.
Simultaneous localization and mapping (SLAM) estimates sensor trajectory while mapping the surroundings. Parallel Tracking and Mapping separates rapid tracking from slower map refinement for small AR workspaces. ORB-SLAM3 supports visual and visual–inertial localization and map reuse [25,147]. These maps locate devices relative to observed structure without defining generated objects’ behavior. Registration aligns virtual and physical structures, accounting for tracking and calibration. Static and motion-dependent errors can make overlays drift or appear detached [6,104].
Optical see-through displays transmit environmental light and add virtual imagery; video see-through displays combine camera images with rendered content. They differ in occlusion, brightness, camera-to-eye alignment, and delay [7]. Placing a virtual object behind a real desk requires aligned surface and depth estimates. Immersive VR controls the entire visible environment but still requires a consistent mapping from head movement to displayed views.

2.2.2. Binocular Perception, Comfort, and Embodiment

Humans recover depth from binocular disparity, motion parallax, occlusion, perspective, and changes in image sharpness. Binocular disparity compares the projections of the same scene point into the two eyes. In the simplified case of rectified pinhole cameras with parallel optical axes, a horizontal baseline, equal focal lengths, and aligned principal points,
d = x L − x R = f px b Z .
Here x L and x R are horizontal pixel coordinates, f px is focal length in pixels, and b is camera separation. Forward depth Z uses the same length units as b. Figure 5(b) shows this geometry: a nearer point produces a larger disparity. This relation applies to corresponding points visible in both images at the same scene time; general eye configurations require calibrated projections. Motion parallax instead concerns relative image movement as the observer moves. Stereoscopic presentation must coordinate these cues with viewing geometry and timing [11]. Interpupillary distance (IPD), the distance between pupil centers, informs the eye separation used for rendering; a mismatch can alter perceived scale or depth. Generated stereo must also handle regions visible to only one eye, where paired correspondence is unavailable.
Vergence rotates the eyes toward a common fixation point; accommodation changes the eye’s optical focus. In natural viewing, both normally correspond to the fixated object’s distance. A conventional fixed-focus stereoscopic display can depict an object at a virtual distance that differs from the display’s optical focal distance. This vergence–accommodation conflict (VAC) can impair binocular performance and cause visual fatigue. Its effects depend on viewing distance and the direction of the conflict [102,265]. Geometric stereo consistency and comfortable presentation are therefore related but distinct. Matching the two images does not change the headset’s physical focal behavior.
Multiple focal planes and gaze-contingent adaptive focus address this mismatch through different optical and computational trade-offs [2,218]. Their suitability depends on the viewing task and observer [152]. Generation can influence disparity, depth ordering, temporal stability, and content placement; the display determines the resulting optical stimuli. Evaluation must therefore describe the headset and viewing geometry alongside the generated content, particularly for near-object manipulation.
Immersion describes system sensory and interaction capabilities, whereas presence concerns the experience of being in the represented environment [271]. Place illusion concerns being somewhere; plausibility concerns whether unfolding events feel real [270]. Presence is a perceptual and behavioral phenomenon, rather than a synonym for image realism [250]. A meta-analysis distinguishes contributions from tracking, stereoscopic presentation, field of view, and other system properties [47].
Embodiment concerns users’ relationship with represented bodies, including self-location, movement agency, and body ownership [143]. First-person virtual-body experiments show that body experience depends on presentation and coupling to the participant [272]. Generated hands or avatars must therefore couple appearance and movement to user actions and nearby objects. Reviewed studies associate presence with cybersickness and suggest possible shared perceptual causes [328]. Visual quality, bodily control, presence, and discomfort require separate measurements.

2.2.3. The Interaction Loop and Its Time Scales

An XR application repeatedly senses the user, estimates pose, interprets input, updates scene state, renders eye views, and presents them. Motion-to-photon (MTP) latency measures the delay from physical movement to its displayed consequence, including sensing, computation, transport, scheduling, rendering, and display contributions [74]. For paired input and response events, distinguish
L k = t k vis − t k in , f out = N − 1 t N out − t 1 out .
The timestamps t are wall-clock measurements. The first expression pairs input event k with its corresponding visible response; the movement and response-detection rule must be specified. The second measures the average rate of N > 1 output events. Its inverse is the mean interval between outputs, not their response delay; Figure 5(c) separates these measurements. The output endpoint must also be named: generated frames, rendered frames, and display presentations can have different rates. Asynchronous time warp uses newer tracking information to transform an already rendered stereo pair near presentation [327]. Such reprojection reduces apparent viewpoint delay but does not execute a new object interaction or provide every surface revealed by a large viewpoint change.
Perceptual thresholds depend on the motion and protocol being tested. Mania et al. found latency discrimination largely unchanged across tested scene-complexity conditions for trained observers. Jerald and Whitton related latency thresholds to scene motion induced by head turns [130,197]. Controller-motion measurements further show that prediction produces different delays at movement onset and during predictable motion [326].
Selection identifies objects; manipulation changes their position, orientation, or scale; locomotion changes the user’s virtual location. Go-Go extends reach through nonlinear mapping, while remote-manipulation studies distinguish object acquisition from subsequent control [22,230]. Body-relative techniques exploit proprioception, the sense of body and limb configuration [205]. Redirected walking changes the physical-to-virtual movement mapping within experimentally studied perceptual limits [282]. These mappings belong to the interface and inform interpretation of generative systems’ control claims.
The instruction “make this chair taller” requires resolving an entity, preserving accepted placement, modifying geometry, and updating the display. A later grasp requires identifiable, selectable or collidable content that follows the intended manipulation. Collaborative use adds compatible object state and awareness of others’ actions [65]. Mini-Me adapts a remote collaborator’s avatar to communicate gaze and gestures across AR and VR viewpoints [225].
Display and interaction fidelity can affect task performance and subjective experience [198]. Presence questionnaires require constructs and administration suited to the study question [256,332]. Comparisons across real and virtual settings further show that scores require interpretation within their protocol [301]. Visual output, retained application state, and participant task outcomes thus describe different stages of the interaction loop.

2.3. Spatial Representations and Rendering

We distinguish generating content from reconstructing observations, then introduce geometric representations and the rendering operations that turn them into images.

2.3.1. Generating and Reconstructing Spatial Content

A visual generator may produce an image, a set of views, or spatial content renderable from new viewpoints. Walking around a stationary chair reveals occluded surfaces while the chair remains fixed. Rendering predicts an observation from a scene and camera; reconstruction infers scene properties from observations. Depth measurements can be fused into a shared volume [48], whereas a single photograph leaves hidden surfaces underdetermined. Learned image-conditioned methods supply additional information: pixelNeRF conditions a radiance field on features learned across scenes, while IBRNet combines features from available source views [316,375].
We distinguish shared model parameters θ from parameters φ fitted to one asset or scene. A trained decoder D θ maps a latent code z to content. Reconstruction may fit φ against observations or infer content using a model learned across scenes [24,192]. A condition c supplies a prompt, reference image, or other constraint. Reconstruction seeks an explanation of observations; generation samples content satisfying a request. Both can use learned priors, so the presence of a neural network does not distinguish these tasks.
Camera parameters v contain intrinsics and extrinsics; physical time τ indexes scene evolution. In Eq. (5), changing v at fixed A φ ( τ ) changes viewpoint; advancing τ at fixed v reveals temporal change. D-NeRF and K-Planes incorporate temporal variation [73,231], but time-indexed representations alone do not define responses to arbitrary new actions.

2.3.2. Geometry: Surfaces, Samples, and Continuous Fields

Geometry describes the spatial extent of matter or the location of surfaces. A triangle mesh stores vertex positions V and triangular connectivity F; a point cloud retains positions without connectivity. Poisson surface reconstruction recovers a global surface from points and estimated orientations [140]. A voxel grid partitions space into cells; doubling resolution along all three axes multiplies the cell count by eight. Sparse storage reduces this burden when much of the volume is empty.
Implicit representations specify surfaces through functions queried at spatial locations x . In a shared decoder parameterization with shape code z, the alternatives can be summarized as
M = ( V , F ) , V = { x j } j = 1 n v , F ⊆ { 1 , … , n v } 3 , S z occ = { x : o θ ( x , z ) = η } , o θ ( x , z ) ∈ [ 0 , 1 ] , S z sdf = { x : f θ ( x , z ) = 0 } .
Each row of F indexes a triangle in the mesh. An occupancy decoder o θ estimates the probability of a point being inside; a chosen threshold η ∈ ( 0 , 1 ) defines its surface [201]. For a closed surface, an exact signed distance function (SDF) has magnitude equal to the shortest Euclidean distance to that surface. Its sign is negative inside under the convention used here. DeepSDF learns an approximation f θ together with shape-specific codes [220]. Its zero level set defines the predicted surface, but fitting a neural field does not guarantee exact distances everywhere. Occupancy values, distances, and mesh connectivity support different computations even when describing similar shapes.
The word “neural” describes a function’s parameterization, not its physical quantity. Networks can predict occupancy or distance, with local features adding detail as in Convolutional Occupancy Networks [224]. Fourier features and SIREN’s periodic activations help coordinate networks represent rapidly changing signals [269,292].
Implicit surfaces can enter mesh pipelines through field sampling and triangle extraction. Marching Cubes constructs triangles where a chosen scalar value crosses grid cells [189]. Sphere tracing instead renders distance fields directly, advancing rays by distance-based steps whose valid bounds prevent surface crossing [92].
A neural radiance field (NeRF) represents density σ φ ( x ) ≥ 0 and direction-dependent RGB radiance r φ ( x , d ) . Here d is a unit viewing direction and φ contains the fitted scene parameters [203]. Density determines ray attenuation, rather than occupancy probability or signed distance; Eq. (6) gives its pixel contributions. However, density alone does not identify a unique watertight surface. NeuS and VolSDF incorporate signed-distance geometry into volume rendering to recover surfaces more directly from images [315,370]. Photographic reconstruction quality and the quality of a collision mesh therefore remain distinct.
Gaussian splatting stores primitives with centers μ i , covariance matrices Σ i , opacities β i , and view-dependent appearance [142]; covariance specifies each Gaussian’s extent and orientation. Projected footprints are blended as in Eq. (7); the primitives need not form a connected surface mesh. Alternatives include oriented planar elements in 2D Gaussian Splatting, surface alignment and mesh extraction in SuGaR, and anchor-organized, view-adaptive Gaussians in Scaffold-GS [86,111,191].

2.3.3. Appearance, Rendering, and Learning from Images

Pixel color depends on geometry, material, lighting, visibility, and viewing direction [119,120,134,135]. Texture maps attach spatially varying values, such as base color or roughness, to surfaces through texture coordinates; they may also contain baked shadows. A bidirectional reflectance distribution function describes how incident light is redistributed toward outgoing directions; Cook–Torrance provides a classic physically motivated model [46].
The rendering equation expresses the interaction between emitted light, incoming illumination, and surface reflection [137]. For this review, a compact interface is sufficient:
I v , τ = R A φ ( τ ) , v , ℓ .
Here I v , τ is the rendered image, and A φ ( τ ) specifies asset or scene geometry and appearance at physical time τ . The camera is specified by v, illumination by ℓ, and the rendering procedure by R . For static content, this representation does not change with τ . For radiance stored under fixed lighting, illumination may already be absorbed into A φ rather than exposed as a controllable input.
Rasterization projects surface primitives and determines pixel coverage, whereas ray-based rendering follows lines of sight. For a NeRF, sample a camera ray at positions x i = o + q i d . Here o is its origin and q i is distance along the unit direction d . Discrete volume rendering gives [203]
I ^ v ( u ) = ∑ i = 1 N T i α i r φ ( x i , d ) + T N + 1 I bg , α i = 1 − exp [ − σ φ ( x i ) Δ i ] , T i = ∏ j < i ( 1 − α j ) .
Here Δ i is the sample interval along the ray, α i is that interval’s opacity, and T i is the fraction transmitted through preceding intervals. Thus T i α i weights the sample’s radiance. The residual transmittance T N + 1 weights the background color I bg ; the empty product is one. Density has inverse-length units, so σ i Δ i is dimensionless. The weights couple visibility to pixel color.
Gaussian splatting uses a related compositing rule but computes opacity from projected primitives rather than ray-sampled density [142]:
I ^ v ( u ) = ∑ i = 1 N u T i ( u ) α i ( u ) r i ( d i ) + T N u + 1 ( u ) I bg , α i ( u ) = β i G i 2 D ( u ) , T i ( u ) = ∏ j < i [ 1 − α j ( u ) ] .
The N u splats overlapping pixel u are indexed in the renderer’s depth order. G i 2 D is the unit-peak Gaussian footprint obtained by locally projecting its 3D covariance, without normalization to unit integral. Learned opacity is β i ∈ [ 0 , 1 ] , and r i ( d i ) is directional RGB appearance. These equations give the basic blending rule; practical rasterizers also filter, threshold, and approximate ordering.
Differentiable rendering estimates how pixels change with scene parameters, allowing an image loss to update a 3D representation. Neural 3D Mesh Renderer introduces approximate rasterization gradients, while IDR jointly estimates implicit geometry, appearance, and camera parameters [139,369].
Inverse rendering seeks scene components that explain the observed appearance. NeRFactor separates albedo, reflectance, visibility, and illumination; nvdiffrec jointly extracts geometry, materials, and lighting [208,393]. Ref-NeRF structures view-dependent reflected radiance to better represent glossy surfaces [305]. Textures and materials require separate treatment because shading recomputed under changed lighting differs from colors stored under the original illumination.
Storage and sampling also influence rendering cost and image quality [309,310]. Plenoxels optimizes a sparse voxel radiance field without a neural network, whereas TensoRF factorizes a field into compact tensor components. Instant-NGP uses multiresolution hash features with small neural networks [30,72,210].
Pixels cover finite areas; ignoring their footprints can cause aliasing under camera movement or resolution changes. Mip-NeRF integrates features over conical ray regions, Mip-NeRF 360 extends to unbounded scenes, and Zip-NeRF combines scale-aware sampling with grid acceleration [13,14,15]. Mip-Splatting handles analogous scale changes through spatial smoothing and image-space filtering [381].

2.4. Generative Learning, Sampling, and State

Generative mechanisms determine how these representations are learned, sampled, and updated. Distribution learning supplies reusable priors, sampling produces content under conditions, and temporal models determine how history and actions influence subsequent outputs.

2.4.1. Learning Visual Distributions and Latent Representations

A visual generator models a distribution p θ ( Y ∣ c ) over outputs Y, given conditions c and shared parameters θ learned from training data. The output may be an image, a video, or a representation rendered by another component. A wooden-chair prompt constrains category and material while leaving exact shape open; sampling produces one realization of that uncertainty. Conditions therefore restrict the set of possible outputs without necessarily specifying them completely [121].
Many models generate through a latent variable z. A variational autoencoder (VAE) learns an encoder distribution q θ E ( z ∣ Y ) and a probabilistic decoder p θ D ( Y ∣ z ) . Its standard unconditional objective maximizes the evidence lower bound (ELBO) [146,245]:
L VAE ( Y ) = E q θ E ( z ∣ Y ) [ log p θ D ( Y ∣ z ) ] − KL q θ E ( z ∣ Y ) ∥ p ( z ) ≤ log p θ D ( Y ) .
The first term rewards reconstruction, and the Kullback–Leibler divergence regularizes inferred codes against a chosen prior p ( z ) . The encoder and decoder parameters θ E , θ D are shared across examples; they differ from the instance parameters φ introduced earlier [117,338]. Sampling draws a code from the prior and decodes it. A latent variable need not compress the data: invertible normalizing flows such as Real NVP retain dimensionality while permitting exact density evaluation [59]. Latent diffusion instead compresses images with a perceptual autoencoder before learning a generative prior [247]. The prior controls which codes are sampled, while the decoder limits the details they can express.
A generative adversarial network (GAN) learns a generator G θ G through competition with a discriminator C θ C  [78]. In the original conditional minimax formulation, both networks receive c [206]:
min θ G max θ C E ( Y , c ) ∼ p data [ log C θ C ( Y , c ) ] + E c ∼ p data ( c ) , z ∼ p ( z ) [ log ( 1 − C θ C ( G θ G ( z , c ) , c ) ) ] .
The discriminator scores whether a sample comes from the training distribution, and the generator maps noise and a condition to an output. The discriminator supplies the training signal; inference uses the generator. StyleGAN additionally organizes variation across synthesis scales [138].
Conditions supplied to a generator include labels, text, images, and spatial maps. Contrastive Language–Image Pre-training (CLIP) learns aligned image and text representations, while Imagen uses a pretrained language model to encode prompts [236,248]. Camera parameters or depth provide further constraints when accepted by the generator. Discrete representations select entries from a learned codebook: VQ-VAE learns such codes, and VQGAN combines perceptual compression with a transformer prior [66,216]. Tokens can encode image or video components without identifying named objects. Token histories therefore differ from XR object records.

2.4.2. Denoising, Probability Transport, and Sampling

Denoising diffusion probabilistic models (DDPMs) learn to reverse a process that progressively corrupts data [99,273]. For a clean representation z 0 , a noisy training input is
z s = α s z 0 + σ s ϵ , ϵ ∼ N ( 0 , I ) , s ∈ [ 0 , 1 ] .
Here s is diffusion time, α s and σ s control signal and noise strength, and I is the identity covariance. The clean endpoint has α 0 = 1 , σ 0 = 0 ; the terminal distribution is chosen to approximate standard Gaussian noise. A common training objective and a reverse step on a discrete DDPM sampling grid are
L den ( θ ) = E z 0 , c , s , ϵ w ( s ) ∥ ϵ θ ( z s , s , c ) − ϵ ∥ 2 2 , p θ ( z s − Δ s ∣ z s , c ) = N μ θ ( z s , s , c ) , Σ θ ( z s , s , c ) , ∇ z s log q s ( z s ∣ c ) ≈ − ϵ θ ( z s , s , c ) / σ s , σ s > 0 .
Training samples data–condition pairs, noise levels, and Gaussian noise; w ( s ) weights noise levels and equals one in the simplified DDPM objective. The predicted noise parameterizes the reverse mean μ θ ; the covariance Σ θ can be fixed or learned. Sampling starts near s = 1 and takes successive steps Δ s > 0 toward s = 0 . The last line links noise prediction to the score of the noisy data marginal q s : a local gradient in representation space. Score-based models use it in reverse stochastic dynamics or an associated probability-flow ordinary differential equation (ODE) [277]. The sampled clean representation is decoded as Y = D θ ( z 0 ) ; this decoder is the identity for pixel-space diffusion. Diffusion time s is distinct from physical time τ in a video.
Flow matching trains a velocity field directly [180]. A linear-path example, closely related to rectified flow [185], uses the same clean-to-noise orientation as Eq. 10:
z ˜ s = ( 1 − s ) z 0 + s ϵ , L CFM ( θ ) = E z 0 , c , ϵ , s ∥ u θ ( z ˜ s , s , c ) − ( ϵ − z 0 ) ∥ 2 2 , d z s d s = u θ ( z s , s , c ) , z 1 ∼ N ( 0 , I ) , s : 1 ⟶ 0 .
During training, a data code z 0 and independent Gaussian noise ϵ define a known interpolation and target velocity; s is sampled uniformly. Conditional flow matching refers to these endpoint-conditioned training paths; external context c additionally conditions the learned field. At inference, the ODE follows the fitted field from noise to data without access to the training endpoint z 0 . Straight training paths need not yield straight sampled trajectories; rectification can improve numerical efficiency. This velocity changes a representation during synthesis rather than describing physical object motion.
The model and its sampler are separate choices. DDIM constructs alternative paths compatible with DDPM training, while DPM-Solver integrates the diffusion ODE with a dedicated higher-order method [190,274]. Progressive distillation trains a student to replace successive teacher steps [249]. Consistency models map trajectory points to a common clean endpoint and support one-step or multistep generation, with distillation or independent training [278]. These procedures change the sampling work that contributes to the response costs examined later.
Guidance modifies sampling to emphasize a condition: classifier guidance uses classifier gradients, whereas classifier-free guidance combines conditional and unconditional predictions [57,98]. Adjusting guidance strength trades diversity against fidelity to the supplied condition. Architecture is a separate choice: a diffusion transformer (DiT) operates on latent patches. Scaled rectified-flow transformers exchange information between text and image tokens [67,223].

2.4.3. Autoregression, Temporal Context, and World State

Autoregressive generation decomposes a joint distribution into ordered conditional predictions:
p θ ( z 1 : N ∣ c ) = ∏ i = 1 N p θ ( z i ∣ z < i , c ) .
Here N is sequence length, z i is token i, and z < i denotes preceding tokens. PixelRNN predicts pixels sequentially; transformers combine tokens through learned attention [215,304]. This prediction order may follow spatial positions or physical time. MaskGIT instead refines masked token sets bidirectionally; continuous-token autoregressive models can train each conditional with diffusion [29,164]. Autoregression orders prediction units, while diffusion can model and sample each unit’s conditional distribution, including frames or chunks in Section 3.3.
Interaction conditions prediction on actions. Let k index rollout steps, o k be an observation, and a k the action applied between steps k − 1 and k. An observation generator can predict p θ ( o k ∣ H k − 1 , a k , c ) directly from a retained history H k − 1 of observations and actions. For fixed external context c, suppressed below, a latent state-space model summarizes the past in z k − 1 state and separates transition from observation generation:
z k state ∼ p θ ( z k state ∣ z k − 1 state , a k ) , o k ∼ p θ ( o k ∣ z k state ) .
The state may contain deterministic recurrent memory and stochastic variables. PlaNet learns such transitions for planning from pixels, and Dreamer learns behavior through imagined latent trajectories [87,88]. The training target determines what the state is encouraged to retain [112]. MuZero predicts reward, policy, and value for planning, while V-JEPA 2 learns predictive embeddings and post-trains an action-conditioned variant for robotic planning [5,255].
Explicit application state exposes object identities, transforms, or geometry to queries and updates. A predictive latent state can support useful rollouts without exposing these quantities. Observation history can preserve visible context without specifying how an accepted object edit persists. Section 3.3.5 examines how these mechanisms store information and update it.

3. Methods: Assets, Scenes, and Dynamic Worlds

XR content generation involves three decisions: which structure an asset retains, how a scene organizes or extends it, and how a world responds to time and actions. We compare conditioning, representations, and update operators to explain their consequences for use. Table 1 catalogues methods in three domain-specific parts; Table 7 records documented XR integrations.
Reading the catalogues. Input tags can indicate alternative conditioning modes. Stage tags identify preparation before use (P), interactive authoring including asynchronous completion (A), or generation during an ongoing rollout (R). They describe workflow placement, not speed; rendering prepared content is separate. An arrow opens a verified official project or GitHub page; a dash indicates that no such page was verified.
The equations use the notation from Section 2: c txt and c img denote text and image conditions, and k indexes construction or rollout steps. A distribution p θ denotes the sampler-induced distribution without assuming a tractable likelihood. Operator chains describe retrieval, optimization, or geometric fusion. Camera queries render existing content; camera conditions guide its generation.

3.1. Generative Models for Visual Assets

Asset generation must infer missing shape and appearance while determining what can later be edited independently. We follow these decisions from 3D construction to materials, parts, and delivery. Let A denote an asset and A φ its representation when fitted per instance; Table 1, Part A, catalogues the methods.

3.1.1. From Image Priors to Native 3D Generation

The first choice is where missing 3D structure is inferred. Image-guided optimization fits an asset through rendered views; view-based methods reconstruct it from generated observations; native 3D models sample spatial variables learned from 3D data.
Representation priors and surface construction. Early learned 3D priors differed in their geometric output. 3D-GAN generated volumetric shapes, latent point-cloud models included learned autoencoders and Gaussian mixtures, and PointFlow separated variation across shapes from variation among their points [1,333,361]. Text alignment requires a semantic condition or objective beyond these geometric distributions.
Surface parameterization and extraction connect these representations to mesh outputs. AtlasNet generates parametric surface patches [81]. DMTet combines deformable tetrahedral grids with differentiable surface extraction; FlexiCubes adds control over isosurface geometry and connectivity during gradient-based optimization [260,261]. Training supervision can also differ from the generated output: GET3D learns textured mesh generation from images, whereas EG3D learns a hybrid representation for 3D-aware image synthesis [26,76].
Semantic guidance and per-asset optimization. Dream Fields optimizes rendered views toward a text description in CLIP’s embedding space [129]. Text2Mesh applies semantic guidance to the colors and local geometry of a supplied mesh, while CLIP-Mesh optimizes a surface together with texture and normal maps [202,207]. A supplied mesh provides a starting structure, but local deformation means that its exact silhouette need not remain fixed.
DreamFusion keeps a pretrained image diffusion model fixed and optimizes an individual 3D representation through differentiable rendering [229]. Let I v = R ( A φ , v ) denote a rendered image at camera v, with other rendering settings suppressed. A noisy image and the score distillation sampling (SDS) gradient are
I v , s = α s I v + σ s ϵ , ϵ ∼ N ( 0 , I ) , g SDS ( φ ) = E v , s , ϵ w ( s ) ∂ I v ∂ φ T sg ϵ ^ θ ( I v , s , c txt , s ) − ϵ .
Here s is diffusion time; α s and σ s define the noise schedule, while w ( s ) weights noise levels and absorbs schedule-dependent factors. The guided noise predictor ϵ ^ θ supplies the image-space residual, which the stop-gradient operator sg holds fixed during differentiation. The update φ ← φ − η g SDS therefore differentiates through the renderer but not the denoiser; η is the step size. Equation (15) describes the prior-guided gradient; method-specific regularizers are handled separately. DreamFusion instantiates A φ as a density-and-albedo field rendered with explicit shading [229].
Magic3D first optimizes a sparse hash-grid field, then refines a textured mesh through differentiable rasterization [171]. Its fine stage applies diffusion guidance to an encoded rendering z v = E θ ( I v ) , adding the image encoder E θ to the gradient path through its Jacobian. ProlificDreamer models a distribution of 3D parameters in score distillation; DreamGaussian optimizes Gaussian primitives and subsequently converts them to a textured mesh [295,321].
Generated views and reconstruction. Camera-conditioned image priors provide evidence about unseen surfaces. Zero-1-to-3 predicts an object view from an image and relative camera change; SyncDreamer couples multiple generated views, and MVDream learns a multiview diffusion prior [184,186,264]. Zero123++ similarly targets consistent image sets. Wonder3D and Unique3D additionally generate surface-normal cues that guide mesh reconstruction [188,263,336]. Consistency in these intermediate predictions matters because a later stage must consolidate them into one surface.
Magic123 fits a per-object field and mesh using reference-view supervision with 2D and 3D diffusion priors [233]. LRM instead maps an image through shared weights to three coordinate-aligned feature planes, forming a triplane radiance representation [106]. One-2-3-45 and InstantMesh reconstruct surfaces or meshes from generated views using learned reconstructors [182,354]. LGM and GRM reconstruct Gaussians from multiview images, including generated views [293,358]. One-2-3-45++ uses the views to condition native 3D diffusion [183].
Native 3D distributions. Other methods learn distributions directly over encoded spatial variables. LION uses global and point-structured latents; SDFusion encodes signed-distance geometry with multimodal conditions; MeshDiffusion models deformable tetrahedral grids [43,187,384].
A learned sampler draws latent content while keeping shared model parameters fixed:
z ∼ p θ ( z ∣ c ) , A = D θ ( z ) , p θ ( P , Z ∣ c ) = p θ ( P ∣ c ) p θ ( Z ∣ P , c ) ( TRELLIS ) .
The first line summarizes sampling and representation-specific decoding. For TRELLIS, z = ( P , Z ) contains active voxel positions P and their local features Z. Separate rectified-flow models generate sparse structure and then its features, conditioned on text or an image [344]. Different decoders map these structured latents to meshes, radiance fields, or Gaussians. Locality supports regional editing, while factorization permits detail variation with coarse structure held fixed. Learned sampling and decoding replace repeated fitting of φ , although sampling can still require multiple numerical steps. Reconstruction metrics test the representation and decoder; generation metrics also test the sampler and condition adherence.
Other native models make different representation choices: Shap-E generates parameters of implicit functions, LN3Diff models latent neural fields, and 3DTopia-XL generates primitives encoding geometry and appearance [41,136,153].

3.1.2. Geometry, Appearance, and Materials

Separating shape from appearance. Once geometry is generated, appearance must align with its surfaces and support the intended rendering controls. Let M denote mesh geometry and μ its appearance attributes, ranging from a texture to explicit material channels. Geometry-first pipelines expose the dependency
p θ ( M , μ ∣ c img ) = p θ ( M ∣ c img ) p θ ( μ ∣ M , c img ) , A = ( M , μ ) .
This factorization separates the initial output from the quantities it constrains. In Hunyuan3D 2.0, image-conditioned shape latents decode to geometry. Rendered normal and position maps then condition multiview image synthesis, followed by texture baking [395]. The second factor thus summarizes several operations rather than a direct material decoder. Shape reconstruction, condition alignment, and texture assessments evaluate different components of this chain.
TRELLIS.2 represents local geometry and physically based rendering (PBR) attributes in O-Voxel, including base color, metallic value, roughness, and opacity [345]. Three stages generate sparse occupancy, geometry latents, and material latents. The material model conditions on the image and generated geometry latents. Separate shape and material autoencoders preserve spatial alignment. Unlike Hunyuan3D’s multiview synthesis and baking, material generation operates directly in native 3D latent space. O-Voxel’s flexible dual-grid surface representation also accommodates open and non-manifold geometry.
Both routes support appearance generation on supplied geometry, holding M fixed while changing μ [345,395]. Explicit material channels provide inputs for changing illumination. The delivered appearance must be assessed under the application’s renderer and lighting.
Appearance representation also determines the available editing operation. Seed3D 2.0 combines coarse-to-fine geometry generation with a unified mixture-of-experts model for multiview albedo and metallic–roughness maps. Part decomposition and articulation construction are additional pipeline stages [82]. MaterialApprentice retrieves expert construction traces, synthesizes a process, and compiles it into shader operations to produce an executable Blender material graph [85]. The former delivers explicit material channels; the latter retains the construction program for subsequent edits. Their evaluations accordingly emphasize rendered-asset preference and material-generation quality or expert editing effort.
Texturing and relighting. TEXTure and Text2Tex condition texture generation on supplied geometry and consolidate observations across viewpoints [32,246]. Fantasia3D instead optimizes geometry and appearance with distinct components, including a physically based material representation [36]. The distinction is which quantities may change: painting an existing surface and synthesizing its geometry are different authoring operations.
A texture may also contain illumination that should change with the environment. Paint3D targets textures without embedded illumination through multiview generation and UV-space refinement, while SceneTex uses a multiresolution texture field and reference-based attention for consistent indoor-scene appearance [33,385]. Relighting comparisons concern the renderer controls introduced through inverse rendering in Section 2.3.3 [208,393].

3.1.3. Structured and Compositional Assets

Generating addressable parts. Editing individual components requires a different output unit from editing an object’s appearance. PartCrafter generates a set of coordinated meshes from a red–green–blue (RGB) image and a requested part count n:
z 1 : n ∼ p θ ( z 1 : n ∣ c img , n ) , A = { M i = D θ ( z i ) } i = 1 n .
Each z i contains one part’s tokens, decoded in a shared canonical coordinate system. The count n sets the number of token groups, distinguished by learned part identities [175]. Local attention processes each group, global attention exchanges information across groups, and image cross-attention conditions both levels. All parts share flow time s during training. The coupled denoising operator thus generates coordinated parts rather than independently sampling and assembling meshes.
Figure 6 connects joint latent generation to decomposition coherence: colors identify part surfaces in the shoe and armchair examples, allowing part structure to be inspected alongside shape quality.
PartCrafter evaluates reconstruction with Chamfer distance and F-score, and decomposition with pairwise voxel-overlap measures [175]. Part selection and independent transformations remain inferred uses; joints, motion constraints, collision behavior, and persistent state require additional definitions and evidence.
TRELLIS edits regions within its structured representation, whereas PartCrafter delivers separate meshes [175,344].
Generation and decomposition as separate operations. TripoSG uses rectified flow to generate shapes; HoloPart performs generative amodal part segmentation of an existing shape [166,366]. Their combined baseline in Table 3 generates the whole shape before inferring hidden part geometry, whereas PartCrafter jointly generates part structure. PartCrafter reports the comparison values for these complete routes.
Mesh topology and semantic parts. PolyGen models vertex and face sequences; MeshGPT generates mesh sequences from learned local features; MeshAnything converts an input shape into an economical mesh [39,211,266]. These methods structure surface connectivity, while part-level methods define selectable or replaceable components. Animation-ready joints require a separate articulation stage.
Recent part generators introduce controls over resolution, semantic composition, and connectivity. FullPart gives each part a full-resolution grid and conditions generation on part bounding boxes. This permits box edits while retaining unchanged part latents [58]. CubePart accepts an open-vocabulary schema of named parts, making semantic composition explicit beyond PartCrafter’s requested count [400]. MeshMosaic generates local mesh patches autoregressively and assembles them through shared boundaries and global shape features. It addresses connectivity rather than semantic decomposition [357]. CubePart’s qualitative engine interactions use downstream articulation scripts.

3.1.4. Conditioning, Editing, and Asset Preparation

Conditioning and delivery turn these representational choices into authoring operations. Text specifies content, images anchor shape and appearance, and local representations constrain edit location. Magic3D demonstrates prompt-based re-optimization and image/style extensions; TRELLIS supports regional edits [171,344]. Hunyuan3D Studio adds image preparation for sketch-based generation and downstream animation tools around its shape and texture models [395].
Evaluation should match the editing unit: condition adherence for whole assets, preservation of unselected regions for local edits, and part identity and object relations for part operations.
Delivery and attribute preservation. DreamFusion and Magic3D offer mesh exports or downloads [171,229]. Official TRELLIS, Hunyuan3D 2.0, and TRELLIS.2 implementations document export paths, including binary glTF (GLB). PartCrafter exports separate parts, a merged object, and a part-to-file manifest [175,344,345,395]. Conversion can alter retained attributes: TRELLIS uses mesh processing and texture baking; TRELLIS.2’s GLB export defaults to opaque materials. Transparency must therefore be enabled after import where relevant [344,345]. PartCrafter’s separate files retain part identities [175].

3.1.5. Comparison and Implications

Asset selection depends on both the source of missing structure and the required edit unit. Image-prior optimization can yield plausible views despite geometric ambiguity; learned spatial latents and coordinated part meshes expose different regional or component controls [175,229,344]. Delivery must preserve the materials, identities, and geometry required by those controls.

3.2. Generative Models for Spatial Scenes

Scene generation adds two problems to asset construction: organizing objects in space and completing the environment beyond observed views. Let S denote the retained scene. Its construction index k differs from physical time τ : traversing a static scene changes camera v k without requiring object dynamics. Table 1, Part B, catalogues assembly, completion, expansion, and memory methods.

3.2.1. Layout Generation and Asset Assembly

Scene assembly separates spatial arrangement from the geometry placed within it. Procedural systems encode selected spatial rules: extended L-systems generate streets and buildings, while CGA shape grammars refine building masses into architectural detail [209,219]. Residential-layout generation combines learned architectural programs with floor-plan optimization [200]. These examples distinguish structural constraints from the decorative appearance of a scene.
Data-driven approaches learn object selection and placement from scenes. Example-based synthesis models plausible arrangements. Deep Convolutional Priors uses top-down room representations; GRAINS models recursive object groups and relationships [70,162,312]. The representation determines whether context is expressed through spatial occupancy, object groups, or explicit relations.
Let B describe the room boundary and L an assembly of assets A i with transforms T i . A learned layout generator can operate on object attributes o i before assets are retrieved:
L = { ( A i , T i ) } i = 1 N , S = Assemble ( B , L ) , o n + 1 ∼ p θ o ∣ { o j } j = 1 n , B .
For ATISS, o i contains category, size, orientation, and position; room type and floor plan define the generation setting. The model predicts the next object’s attributes from the existing object set and floor-plan features. A permutation-invariant context encoder and sampled object permutations during training avoid imposing a fixed ordering on room objects. Retrieved assets instantiate the resulting labeled boxes [222]. Geometry availability and layout probability are thus distinct parts of construction.
Scene graphs make those relations directly addressable. PlanIT generates a relation graph before spatial instantiation; Graph-to-3D jointly generates object shapes and layouts from a graph; EchoScene couples shape and layout denoising through information exchange [56,313,386]. Editing a graph specifies a change in objects or relations, while the generated geometry and transforms must still realize it spatially.
Language-conditioned systems provide an interface to these structural decisions. LayoutGPT uses in-context demonstrations to produce spatial plans; InstructScene links instructions to a semantic-graph prior and layout decoder [69,170]. Holodeck combines staged large language model (LLM) specifications, asset retrieval, and constraint solving [365]. Queries Q = LLM θ ( c txt ) specify room geometry, object descriptions, and relations. Retrieval selects assets A i ; { T i } = Solve ( B , { A i } , Q ) places them. The solver enforces containment and noncollision while seeking to satisfy soft relations, producing an AI2-THOR scene. Evaluation includes layout plausibility and navigation in NoveltyTHOR environments. Retained asset identities supply units for later authoring, while retrieval limits available shapes.
DiffuScene denoises unordered object attributes, while PhyScene adds collision, room-layout, and reachability guidance to scene synthesis [294,364]. Ctrl-Room connects arrangement and appearance through a layout-conditioned panorama followed by 3D room reconstruction. Its scene representation supports individual furniture edits [68].
Agentic systems incorporate assessment into repeated scene construction. Scenethesis combines image-guided semantic placement with geometric optimization for contact, collision, and stability, followed by scene judging and replanning [179]. SAGE coordinates asset and layout tools with visual and physics critics, using simulation feedback to revise candidate scenes [341]. Compared with Holodeck’s staged constraint solving, these systems change where assessment feeds back into construction. Both evaluate scene plausibility and physical or embodied tasks under different protocols. Scenethesis includes reachability and walkability; SAGE reports short-horizon stability and policies trained from generated demonstrations.

3.2.2. Scene Completion and Spatial Expansion

Incremental surface construction. View-based construction addresses coverage by generating newly revealed content in agreement with a retained surface. The main operation alternates observation synthesis with geometric fusion. With m k = 1 for known projected pixels, a common update is
( I ¯ k , D ¯ k , m k ) = Π ( S k − 1 , v k ) , I k ∼ p θ ( I ∣ I ¯ k , m k , c txt , k ) , D ^ k = AlignDepth k ( I k , D ¯ k , m k ) , S k = U k ( S k − 1 , I k , D ^ k , m k , v k ) .
Here Π projects the retained representation to RGB, depth, and visibility; AlignDepth k combines depth prediction with geometric alignment; and U k unprojects, filters, and merges new content. Projected RGB and visibility condition the image sampler; projected depth constrains depth completion or alignment in AlignDepth k .
Text2Room instantiates S k as a textured mesh M k . Its depth operator completes depth from the known rendered region, then fits scale and shift in disparity space. The update filters stretched triangles and stitches new faces to existing geometry [103]. A two-stage camera/prompt schedule first constructs the room, then selects views exposing remaining holes. Mesh downloads and a browser viewer establish delivery and camera navigation; human judgments assess perceived structure and completeness. Poor viewpoints can leave stretched surfaces or holes; Poisson closure can oversmooth missed regions, and textures retain generated illumination [103].
SceneScape retains mesh projection but changes the refinement operators. At each view, it fine-tunes the depth predictor against D ¯ k over known pixels. It then adapts the image decoder to preserve projected appearance, regularizing new regions toward the original decoder output. Both networks reset to their pretrained weights at each step [71]. The image sampler in Equation (20) is therefore p θ k , where θ k includes the temporary decoder update. Depth adaptation occurs within AlignDepth k ; these network updates differ from fitting scene-content parameters φ . Mesh updates add newly visible content while retaining previously synthesized, possibly occluded surfaces. Camera recovery, reprojection/depth error, and reconstruction tests assess how these updates affect agreement across viewpoints. Incorrect retained surfaces can misguide later synthesis, and large sky–ground depth discontinuities remain difficult for the triangular mesh [71].
Figure 7 illustrates why plausible frames and geometric agreement can diverge. The ablation evaluates 20 videos from 10 prompts. COLMAP provides a reconstruction-based check whose coverage depends on the generated views. Here, temporal continuity concerns camera traversal, not independently moving objects.
Fields and coupled views. A learned spatial field can replace incremental fusion state. SceneDreamer’s core generator uses no external text or image condition. Scene noise z scene procedurally determines a bird’s-eye-view height field h and semantic field m sem , while z style controls appearance. In shared notation, its construction is
( h , m sem ) = BEV ( z scene ) , f scene = E θ ( h , m sem ) , f x = H θ ( x , f scene ) , S ( x ) = F θ ( f x , m sem ( x ) , z style ) , I v = R ( S , v ) .
Here E θ encodes height and semantic fields into the scene feature f scene ; x is a queried 3D position. The generative hash grid H θ indexes location- and scene-dependent features; F θ supplies density and color for volumetric rendering [40]. Shared parameters are learned adversarially from unposed image collections. New cameras query a sampled scene rather than rebuilding it through Equation (20); style changes modify its appearance condition. The height field restricts vertically overlapping structures, including tunnels and caves. Reported trajectory tests therefore concern this class of static landscapes.
MVDiffusion couples generated observations by denoising multiple views with correspondence-aware attention. This replaces sampling each I k from a fused predecessor in Equation (20). Homographies connect panoramic crops; depth maps and camera poses supply correspondences in the geometry-conditioned configuration [297]. Overlap-consistency evaluation tests agreement across views; panoramic angular coverage alone leaves stereo baseline and geometry for viewpoint translation unspecified.
Panorama-to-scene construction. HunyuanWorld 1.0 adds a panoramic intermediate, I pan ∼ p θ ( I pan ∣ c ) , followed by semantic decomposition and layer-wise reconstruction [127]. For text input, c is an enhanced description. Image input supplies a projected image condition and an auxiliary scene description. The intermediate contains foreground, background, and, for outdoor scenes, sky layers. Predicted depths are aligned across overlapping regions before mesh construction. Foreground geometry comes from depth-based sheet warping or an additional image-to-3D stage with object placement. Compared with sequential mesh fusion, the state becomes a layered panorama, and reconstruction preserves separable object components. Reference-centered angular coverage still requires hidden surfaces to be completed or generated. The report describes engine integration and demonstrates object translation, rotation, and scaling; image-alignment measures evaluate the input condition. Its optional long-range extension uses Voyager’s geometric memory, discussed next.
Extending a scene across observations. LucidDreamer expands Gaussian scenes with generated imagery; WonderJourney creates connected visual journeys; WonderWorld targets interactive scene generation from one image [44,376,377]. Invisible Stitch uses depth inpainting to construct newly revealed regions [64]. Overlap with retained surfaces constrains an extension, while unseen regions can admit several plausible completions.
Text2NeRF combines generated imagery, monocular depth, inpainting, and radiance-field updates for text-conditioned expansion [389]. It uses a field rather than surface or Gaussian reconstruction to retain the aligned observations.
Panoramic and layout-conditioned construction. PanFusion couples panoramic and perspective diffusion branches to output a panorama [387]. DreamScene360 reconstructs a Gaussian scene from a panorama, while Pano3DComposer generates and places object components from panoramic input [235,397]. SceneCraft supplies geometry earlier: it renders a specified 3D semantic layout into multiview proxy maps, conditions diffusion on their semantics and depth, and fits a NeRF to the generated observations [363]. Its layout constrains synthesis before reconstruction; panorama-based routes infer geometry and placement from imagery.
Generation and reconstruction can enter at different stages of scene construction. I-Scene adapts a pretrained instance generator to scene RGB images and instance masks through spatial guidance, scene-context attention, and view-centric coordinates. It produces separately represented objects with shared spatial context [178]. SceneFrom3D starts from supplied outdoor geometry and per-object identity images. It schedules anchor views and interpolated video, then fits a Gaussian scene to the generated observations [144]. I-Scene transfers an instance prior into scene organization; SceneFrom3D constrains observation synthesis with an existing layout. SceneFrom3D’s component-substitution comparisons separately test generation, reconstruction, and view scheduling.
For temporal imagery, 360DVD generates motion-conditioned panoramic video and CubeComposer expands perspective video into high-resolution panoramic sequences [161,317]. Looking around a viewpoint uses angular coverage; translation additionally requires geometry or a novel-view mechanism.
SynCity combines pretrained image and 3D generators to condition each new tile on the surrounding world and fuse it with the scene [63].

3.2.3. Camera-Controlled Generation and Geometric Memory

Geometry as generation memory. Repeated expansion makes retained geometry a constraint on later synthesis. Voyager moves depth from a separate estimator into the jointly generated output. Let P k − 1 be the accumulated colored point cloud, v k a requested camera sequence, and q k its projected partial color-and-depth (RGB-D) observations and visibility masks. At clip level,
q k = Π ( P k − 1 , v k ) , ( I k , D k ) ∼ p θ ( I , D ∣ q k , ω k − 1 ) , P k = Cull P k − 1 ∪ Unproject ( I k , D k , v k ) .
Here ω k − 1 denotes overlapping-clip state used during sampling; the initial cache is unprojected from input image c img with estimated depth [123]. Unlike RGB inpainting in Equation (20), the model jointly predicts aligned RGB and depth, adding projected depth as a geometric condition. Culling selects useful new points by visibility and orientation; overlapping-segment initialization and refinement smooth clip transitions. The cache supports subsequent synthesis, while generated observations and depths can also supervise three-dimensional Gaussian splatting (3DGS) reconstruction. Camera-control and spatial-consistency evaluations, including WorldScore’s static subset, test path adherence and agreement among views [123].
Offline world construction. HY-World 2.0 combines panoramic initialization, planned trajectories, memory-guided keyframe expansion, reconstruction, and scene composition offline [298]. Compared with Voyager, its WorldStereo 2.0 expansion component generates keyframes using both global point-cloud guidance and retrieved observations. Memory includes a panoramic point cloud and a bank H k of image–camera pairs. Selected references are paired horizontally with target views; generated keyframes update H k for later retrieval. Camera rays and embeddings provide further viewpoint control. WorldMirror 2.0 and depth alignment subsequently reconstruct this keyframe state, which is distinct from the final Gaussian representation.
Scene composition fits per-instance Gaussian parameters φ to views and geometric supervision, then extracts a collision-proxy mesh through a truncated signed distance field. The output Y = ( S φ , M collision ) thus separates rendering and collision roles. Evaluation likewise distinguishes text/image-conditioned world generation from reconstruction supplied with multiple views or video. Camera errors evaluate WorldStereo 2.0; single-image generative reconstruction is assessed against multiview-stereo pseudo-ground truth [298]. Navigation and stair-collision demonstrations use the subsequently constructed scene and proxy.
View synthesis and persistent reconstruction. Stable Virtual Camera (SEVA) generates camera-conditioned views without explicit reconstruction as its central representation [396]. ViewCrafter guides video diffusion with point clouds; See3D learns multiview generation from pose-free video with visual conditioning [194,380]. FlexWorld integrates generated content into a retained scene, whereas GEN3C uses a 3D cache for camera-controlled generation [35,244].

3.2.4. Persistent Representations and Spatial Functionality

Retained surfaces, addressable objects, and collision proxies expose different spatial operations. Editing methods make the update mechanism explicit.
Editing retained representations. Instruct-NeRF2NeRF updates a radiance field from instruction-edited observations. GaussianEditor couples semantic editing with Gaussians, and SceneFactor factors a latent 3D scene representation for controllable synthesis and modification [21,38,91]. Each retains content for subsequent rendering; later operations depend on whether the addressable unit is an appearance region, geometric component, scene entity, or simulator variable.
Procedural generation and explicit constraints. Infinigen generates natural-scene geometry and appearance procedurally; Infinigen Indoors adds indoor assets and constraint-based arrangement [238,239]. Constraints specify compositions for a solver, while learned priors estimate regularities from examples; PhyScene shows how explicit requirements can also guide learned generation. Both routes require operation-specific assessment, such as keeping passages clear or supporting placed objects.

3.2.5. Comparison and Implications

Scene assembly and incremental completion fail for different reasons: unavailable assets or incompatible layouts constrain the former, while incorrect depth and accumulated surface errors constrain the latter [71,365]. Object rearrangement requires addressable entities; visual walkthroughs require coherent surface coverage.

3.3. Dynamic Content and Action-Conditioned Worlds

Static scene construction determines what exists; dynamic generation also determines what changes and what drives those changes. Prepared 3D dynamics expose camera/time queries, while controlled rollouts predict content from actions and history. Table 1, Part C, catalogues these methods.

3.3.1. Video Priors for Dynamic Generation

From image synthesis to temporal prediction. Independently generated frames can flicker, change identity, or break motion continuity. Video Diffusion Models adds temporal modeling and joint image–video training [101]. Make-A-Video learns appearance and language associations from paired text–image data and motion from uncaptioned videos [267]. Video LDM learns temporal alignment in an image generator’s compressed representation [20]. These approaches transfer image priors while using additional architecture and video training to connect observations across time.
Temporal resolution and the organization of synthesis. Imagen Video distributes spatial detail and frame rate across a base generator and spatial/temporal super-resolution cascade [100]. Lumiere processes the full temporal extent at several space–time scales, generating a full-frame-rate, low-resolution clip within its denoising network [12]. The full clip is refined through iterative denoising. Training organization also differs: Stable Video Diffusion separates image pretraining, video pretraining, and high-quality video finetuning [19]. VideoCrafter2 studies spatial/temporal coupling, then improves appearance with high-quality images while retaining learned motion [34].
Large video backbones and downstream capabilities. HunyuanVideo spans architecture, training, data processing, and execution infrastructure [151]. Wan combines a video VAE with a flow-matching transformer for generation and editing [307]. CogVideoX introduces spatiotemporal compression and expert transformer components for text–video integration [367]. Movie Gen extends foundation-model training to video synthesis, personalization, instruction-based editing, and synchronized audio [228]. These backbones supply visual and motion priors for downstream adaptation. That adaptation must determine how generated edits remain available to later application views and interactions.
Discrete video representations and temporal ordering. VideoGPT combines a video VQ-VAE with a transformer over discrete space–time codes [359]. CogVideo transfers a text-to-image transformer and trains hierarchically across frame rates [105]. Phenaki combines a temporally causal tokenizer for variable-length video with masked-transformer generation under changing text prompts [306]. Tokenizer causality limits which frames contribute to a code; code-generation order is a separate choice. MAGVIT uses masked video-token modeling for multiple synthesis tasks [379], while VideoPoet represents multimodal inputs and outputs autoregressively [149].
Hybrid autoregressive and denoising models. NOVA predicts frames causally while modeling spatial sets in a continuous representation within each frame [55]. Lumos-1 combines a language-model-style architecture with masked discrete diffusion, using bidirectional within-frame and causal between-frame attention [382]. MAGI-1 denoises successive video chunks with noise levels organized for causal temporal generation [251]. These ordered temporal processes differ in token type, prediction unit, and objective. Streaming assessment must identify when each unit becomes available, which previous outputs remain in context, and whether a new instruction can affect the next unit.

3.3.2. Generating Time-Varying 3D Content

To view generated motion from different positions, one route fits a time-indexed 3D representation S φ ( τ ) before playback. A renderer then separates the scene-time query from the camera query:
I ( v , τ ) = R S φ ( τ ) , v .
Varying v at fixed τ produces different views of the same modeled state; varying τ at fixed v exposes its temporal evolution. These queries allow spatial agreement and motion to be evaluated separately. Methods differ in how S φ ( τ ) is parameterized and which priors or reference observations supervise φ . A time coordinate selects a point in the prepared animation; it does not itself supply an action-dependent transition.
4D-fy instantiates S φ ( τ ) as a radiance field with static and dynamic hash-grid features [9]. In common notation, ( σ , r ) = f φ MLP ( h φ stat ( x ) + h φ dyn ( x , τ ) ) gives density and color at position x ; the inspected model omits view-dependent color. Compared with Eq. (15), guidance is staged among multiview SDS, image-based variational score distillation (VSD), and video SDS. Camera extrinsics additionally condition the multiview prior; rendered frame sequences supply the video prior. These supervision operators alternate updates rather than forming a fixed sum of identical SDS losses. Dynamic features are unfrozen for video-guided updates in the last stage. Human evaluation separates appearance, perceived 3D structure, and motion. Against MAV3D, 41% of motion-quality preferences favored 4D-fy, with no statistically significant difference reported for that criterion.
Figure 8 illustrates how guidance and field representation affect shape and detail for two prompts. Its time/viewpoint axes distinguish spatial from temporal agreement. The sampled stills support inspection of appearance and geometry. Motion quality comes from human judgments of the animation.
Deforming a static asset. DreamGaussian4D instead writes the dynamic representation as S φ ( τ ) = W φ ( S 0 , τ ) , where S 0 is a static Gaussian asset and W φ predicts changes in position, rotation, and scale using HexPlane features [243]. Its additional supervision is a driving video { I k drive } k = 1 K at a reference camera v ref :
L ref ( φ ) = 1 K ∑ k = 1 K R W φ ( S 0 , τ k ) , v ref − I k drive 2 2 .
The video may be supplied or generated from the input image. Reference-view fitting is combined with 3D-aware image SDS at novel views; subsequent video-to-video refinement improves mesh-sequence textures. Whereas 4D-fy uses a video-prior gradient, the driving video supplies explicit framewise deformation targets. Evaluation covers video-to-4D novel-view reconstruction and video quality; demonstrations compose textured mesh sequences in Blender.
From dynamic reconstruction to generation. D-NeRF models deformation relative to a canonical radiance field, and K-Planes factorizes spatiotemporal fields into lower-dimensional planes [73,231]. They supply representation and reconstruction mechanisms that later generative systems can use; neither an additional time coordinate nor a deformable field by itself specifies a generative prior. MAV3D learns a dynamic scene from text and video priors, while Consistent4D reconstructs a dynamic object from monocular video using consistency across generated observations [133,268]. This separates the source of motion supervision from the data structure used to replay that motion.
Align Your Gaussians combines dynamic Gaussians and diffusion priors for text-to-4D generation; SV4D generates jointly consistent frames and views before fitting dynamic 3D content [176,349]. The former emphasizes a deformable explicit rendering representation; the latter concentrates consistency modeling in multiview video generation. Full-4D likewise synthesizes time–view observations, using geometric reprojection during video generation and flow-matching distillation when fitting deformable Gaussians [37]. Its comparisons combine delivered 4D scenes with generated-video baselines used when reconstruction is unavailable or fails.

3.3.3. Separating Camera Motion from Object Dynamics

The same camera/time separation must be resolved when generated observations supply the reconstruction targets. DimensionX models these factors through two desired observation families, expressed using Eq. (23) as
V S = { I ( v k , τ 0 ) } k = 1 K , V T = { I ( v 0 , τ k ) } k = 1 K .
The S-Director learns camera variation at fixed scene time; the T-Director learns temporal variation at a fixed camera [284]. In the inspected preprint, separate low-rank adaptations learn the two factors. A switch during diffusion time s applies spatial guidance before temporal guidance. The guidance switch alters the denoising operator, rather than simply appending physical time τ to the input.
Compared with direct optimization of S φ ( τ ) , DimensionX first generates spatially and temporally varying observations from an image. Trajectory-specific S-Directors control the view sequence; the sparse-view route additionally conditions on an ending image. Identity-preserving denoising and temporal refinement connect generated views before 3D or 4D reconstruction [284]. Novel-view reconstruction metrics and subject-consistency assessment test different components. The reconstructed output is a Gaussian representation.
Independent controls for viewpoint and motion. MotionCtrl controls camera and object motion within video generation; CameraCtrl adapts video diffusion to supplied camera trajectories [94,323]. CAT4D conditions multiview video on camera and time, then recovers a deformable scene [337]. These mechanisms distinguish walking around a stationary person from observing that person’s motion with a fixed camera. In XR, tracked user motion continuously changes viewpoint, while object animation or manipulation supplies a separate source of change.
Reference appearance and motion conditions. Tune-A-Video adapts image diffusion to a single text–video example and guides structure through inversion [335]. AnimateDiff learns a motion module transferable to personalized image models derived from the same base [84]; DynamiCrafter conditions video on contextual and direct image information [350]. VideoComposer adds textual, spatial, and temporal conditions, including motion vectors [318]. Distinguishing object from camera motion still requires separate conditioning and evaluation; an animated reference alone leaves the decomposition ambiguous [107,356].

3.3.4. Action-Conditioned Video Generation

Action-conditioned models sample the next observation from the current input and available history. Let V k be the next frame or chunk, a k its action inputs, H k − 1 the retained history with any associated metadata, and C k the context selected for prediction. A common rollout interface is
V k ∼ p θ ( V k ∣ C k , a k , c ) , H k = Update ( H k − 1 , V k , a k ) .
Selection and update may retain observations, latent features, or attention caches. They need not preserve all past information or constitute a complete Markov state. In contrast to Eq. (23), each new action affects the sampling of subsequent content rather than only a query to a prepared representation.
Matrix-Game 2.0 instantiates a k with frame-level keyboard and mouse signals and initializes generation from an image; its inspected architecture removes the text branch [95]. Mouse signals enter through latent concatenation and temporal self-attention, while keyboard embeddings enter through cross-attention. Its fixed-length latent/action cache restricts C k to recent context. Self-Forcing trains the few-step autoregressive student on its own generated histories. The reported temporal, motion, and input-adherence tests assess this configuration; scene/object scores remain perceptual video measures.
WorldPlay uses dual action conditioning a k = ( a k key , v k ) , combining discrete actions with explicit camera parameters [285]. Key embeddings enter the time embedding; camera intrinsics and extrinsics enter an additional projective positional-encoding attention path. Poses also support retrieval of historical observations. Context Forcing aligns the teacher’s and student’s available memory during distillation, without adding a user input. Evaluation distinguishes trajectory-based camera-control measurements from navigation and text-triggered event demonstrations.
Interface-level demonstrations. Google DeepMind’s Genie 3 research release is an industrial counterpart. Its official recordings show navigation, revisiting, weather changes, and new objects, and the release describes agent-provided navigation actions [221]. The release documents interface behavior, with limited architectural and experimental detail for internal-memory comparison.
Generated Reality supplies the tracked-input configuration examined in Section 5 [348].
Action interfaces and learned dynamics. Genie learns latent actions from video without explicit action labels. GameNGen instead conditions the next game observation on recorded game actions and recent history [23,303]. This distinguishes an inferred action encoding from a supplied command vocabulary. DIAMOND also conditions visual prediction on actions, but studies the resulting model as an environment for learning a control policy [3]. Action encoding and the purpose of a rollout are therefore separate choices. An XR integration must map its tracked pose, hand event, or authoring command to the accepted action space.
Hunyuan-GameCraft combines controls with hybrid historical conditioning; GameCraft-2 supports instruction-following interaction and introduces InterBench [160,296]. Image-to-video configurations of DynamiCrafter, EasyAnimate, Allegro, and CogVideoX are also world-generation benchmark comparators [350,355,367,399]. WorldScore’s scores apply to the camera and action conditions of the evaluated image-to-video checkpoints.
Noise levels as a representation of available history. Unlike joint clip denoising, interactive continuation has an observed past and uncertain future. Diffusion Forcing trains with independent noise levels per sequence token, retaining cleaner context while denoising future tokens [31]. Noise levels express generation uncertainty, not elapsed physical time. History-Guided Video Diffusion accepts variable-length history through its transformer and objective and uses that history for sampling guidance [275]. Both make the amount and reliability of context central to prediction.
Causal generation and training on generated histories. CausVid adapts bidirectional video diffusion into a causal generator and distills sampling into few steps [374]. Causal attention enables caching as the sequence grows, reducing work before display. Self Forcing instead addresses the mismatch between clean training histories and imperfect generated histories. It rolls out and supervises generated sequences during training [124]. Architecture and distillation reduce dependencies and sampling work; generated-context training addresses rollout errors. Throughput must therefore be read alongside training context and the evaluated quality horizon.
Extending the generation horizon. FreeNoise reschedules initial noise and uses windowed temporal attention for longer sequences, including changing text conditions [234]. FIFO-Diffusion queues frames at different noise levels, emitting completed frames and adding new noise [145]. StreamingT2V conditions chunks on recent features and an appearance reference from the initial chunk [96]. Correlated noise, recent context, and persistent appearance can retain different information about earlier events. LongLive combines causal generation, streaming long-video training, an attention sink, and cache refresh when prompts change [362]. Revisiting locations and restoring changed objects test spatial recall and retained modifications.
Turning controls into predicted consequences. GameFactory learns keyboard/mouse control separately from game-style adaptation [378]; Pandora combines language modeling and video generation to predict visual states from free-text actions [343]. iVideoGPT places observations, actions, and rewards in a common token sequence for prediction, planning, and reinforcement learning [334]. Vid2World adapts pretrained video diffusion through causalization and action guidance [122]. Responses must be interpreted at their action granularity: language commands and calibrated controller displacements carry different control semantics.

3.3.5. Memory and Persistent Interaction

Retrieving visual history. A recent-only context can lose access to locations left earlier in a rollout. Spatial retrieval addresses this by selecting relevant past observations for Eq. (26). For Matrix-Game 3.0 and WorldPlay, this selection can be described as
C k T = Recent ( H k − 1 ) , C k S = Retrieve ( H k − 1 ; v k ) , C k = Merge ( C k T , C k S ) .
Here Merge denotes model-specific encoding and attention, not necessarily set union. Matrix-Game 3.0 retrieves frames using camera and field-of-view relations and encodes their relative camera information. It jointly attends to retrieved memory, recent context, and current predictions [325]. Error-aware training perturbs historical conditions with prediction residuals. Rollout is autoregressive across segments, while the student retains bidirectional attention within each prediction segment. The retrieval examples test recovery after camera movement and occlusion; the speed ablation tests a separate property.
WorldPlay instantiates C k T with recent chunks and C k S with non-adjacent frames selected by field-of-view overlap and camera distance [285]. Temporal reframing assigns retrieved observations nearby positional indices instead of retaining unbounded absolute gaps. It changes the encoding of selected conditions, not their physical capture times. Figure 9 pairs the selection/append loop with the standard-versus-reframed rotary positional embedding (RoPE) ablation. The round-trip protocol compares returning views with earlier generated views, testing visual recall of the established environment.
Relative to WorldPlay’s frame retrieval, AnchorWeave selects local point-cloud memories by viewpoint coverage and reconciles them through a multi-anchor controller [324]. The geometry serves as generation context (Section 3.2.3). ActWorld changes what history is prioritized: an interaction-aware router retains transitions, while persistent event/object tokens preserve anchors after frames leave the buffer [352]. Geometric coverage and interaction relevance therefore select different evidence for a later observation.
Memory allocation and cost. Additional history can preserve useful context while reintroducing generated errors. Matrix-Game 2.0’s qualitative cache comparison shows earlier artifacts with nine rather than six latent frames [95]. WorldPlay’s fixed-budget comparison favors different spatial/temporal allocations under different consistency and control metrics [285]. Memory selection therefore trades recent continuity against access to older views. Implementation also affects throughput: Matrix-Game 3.0’s asynchronous ablation uses eight graphics processing units (GPUs) for diffusion inference and one for video decoding. This configuration also applies 75% pruning of the video autoencoder. Removing GPU retrieval optimization reduces throughput from about 40 to 6.60 frames per second (FPS) [325].
Shared state and generated observations. MultiGen adds explicit geometric state to the history-conditioned observation model [227]. Let Λ be its static level map, p k − 1 the current player pose, and r k − 1 its disparity readout. Its observation/dynamics split can be summarized as
r k − 1 = RayDisp ( Λ , p k − 1 ) , I k ∼ p θ ( I k ∣ C k , r k − 1 , a k ) , p k = p k − 1 ⊕ Δ θ ( p k − 1 , a k , r k − 1 , f k ) .
Here C k contains recent frames ending at I k − 1 ; a k drives the transition to frame k. The readout is concatenated with context frames, and action embeddings enter the observation network through cross-attention. The dynamics module also uses features f k from denoising I k ; ⊕ applies the predicted pose increment and wraps the angle. Generation is therefore coupled to an explicit state update, beyond adding another history frame. The map is editable before rollout and fixed during it; player states update online and are shared. Simulator-referenced rollouts evaluate observation fidelity; the single-map multiplayer test uses a vision-language model to judge opponent presence.
Solaris jointly denoises two Minecraft views under separate player actions, coupling their streams through cross-player self-attention [253]. This differs from MultiGen’s pose updates against an authored map. Building and cross-view tests with two players extend multiplayer evidence beyond opponent presence.
Predictive state and its intended use. DreamerV3 learns state for control: its training and normalization support behavior learning across diverse tasks [89]. As distinguished in Section 2.4.3, state sufficient for action selection need not retain every display detail; a detailed frame need not recover earlier object interactions.
View memory and recurrent context. VMem retrieves spatially relevant views through a surfel index [163]. Video World Models with Long-term Spatial Memory retains geometrically grounded information; Long-Context State-Space Video World Models extends temporal context with recurrent state-space processing [226,339]. ReWorld combines a bounded key–value cache with a pose-indexed landmark bank for revisits [42]. Geometric indices, chronological caches, and recurrent summaries provide different access to earlier observations.
Retaining changes in a scene. WorldMem associates frames with poses and timestamps. It demonstrates object placement and tests temporal conditioning on placing-event videos [346]. PERSIST learns an evolving latent 3D environment specified or edited in three dimensions [77]. FloWM structures memory into velocity channels transformed by self-motion and object flow, with tests in simplified dynamic worlds [168]. LiveWorld instead keeps entity monitors active outside the observer’s view and uses their evolved states on return [62]. These approaches distinguish recalling changed observations from predicting changes during absence.
Programmable World Model separates state transitions from visual generation. External rules update entity attributes and relations, which the engine compiles into generator conditions [126]. These transitions are programmed rather than learned as latent dynamics. WorldDirector augments appearance retrieval with planned 3D object trajectories and identity-specific conditions for disappearance and re-entry [311]. Trajectory orchestration alone does not establish physical simulation. From Pixels to States supplies an action–state–observation vocabulary for this comparison [167]. Section 6.3 distinguishes accepting a change from retaining it after departure and return.

3.3.6. Comparison and Implications

Prepared dynamics, history-conditioned observations, and explicit state updates support different tasks: animated exhibits, visual exploration, and observers linked by shared state. MultiGen, for example, reports that textures and small objects absent from its map are not explicitly preserved on revisiting [227]. Evaluation should measure retention of the task-relevant variable.
Table 1. Catalogue of visual-asset, spatial-scene, and temporal generative methods.
Table 1. Catalogue of visual-asset, spatial-scene, and temporal generative methods.
Preprints 240396 i001
Preprints 240396 i002
Preprints 240396 i003
Preprints 240396 i004
Preprints 240396 i005
Preprints 240396 i006
Preprints 240396 i007

4. Data and Evaluation

Evaluation connects model capabilities to XR task requirements through data roles, measured endpoints, quality, and operating costs. Table 2 summarizes supporting resources; Figure 10 maps capability coverage in the detailed evidence subset. These results inform the system analysis and task-based selection in Section 5.

4.1. Data Sources and Their Roles

Object collections supervise geometry and appearance, scene observations support spatial reconstruction, and executable environments expose actions and outcomes. WorldPlay uses DL3DV among its training sources, while Generated Reality uses HOT3D for egocentric hand–object interaction [285,348]. HOT3D provides synchronized egocentric views with tracked hand, object, and camera poses [10]. Figure 9 and Figure 13 illustrate the datasets’ capture contexts.
Test data define the reference a result can recover. SceneScape evaluates 22 indoor RealEstate10K videos subsampled to 25 frames. Under the same prompt, SceneScape receives camera poses and GEN-1 receives RGB frames [71]. Holodeck’s artist-designed NoveltyTHOR environments instead support navigation evaluation [365].
Table 2 describes public resources and their scales; Table A1 records the source-specific experimental subsets and workloads. Counts retain their original units, including objects, trajectories, and user ratings.
Geometry, appearance, and their capture. ShapeNet, Objaverse, and Objaverse-XL provide object models with different category coverage and annotations. ABO adds product-centric images, models, and benchmarks [28,45,53,54]. Google Scanned Objects supplies textured household models, while Common Objects in 3D (CO3D) couples object-centric videos with reconstructed geometry [60,242]. Filtering, rendering, licensing, and split decisions determine the subset used for shape and appearance learning; resource totals therefore differ from effective training size.
Scene observations and reconstructed structure. 3D-FRONT supplies furnished room structure, DL3DV-10K supplies scene observations, and RealEstate10K supports posed video-based view synthesis [75,177,398]. SUN RGB-D provides color images, depth measurements, and scene annotations. The Stanford Large-Scale 3D Indoor Spaces (S3DIS) dataset annotates building interiors. ScanNet links camera trajectories, RGB-D observations, reconstructed surfaces, and semantics [4,49,276]. Matterport3D and Replica emphasize reconstructed indoor environments, while ScanNet++ and ARKitScenes cover different capture devices and geometry/appearance conditions [16,27,283,373]. To distinguish view agreement from recovery of unobserved structure, evaluation must identify observed versus completed surfaces and the reference type: imagery, geometry, or annotation.
Executable environments and task state. Habitat-Matterport 3D (HM3D) provides reconstructed spaces that can support simulation. Habitat and Gibson expose environments for embodied navigation [240,254,340]. AI2-THOR adds object actions, ProcTHOR scales scene construction, and Habitat 2.0 supports rearrangement [52,148,289]. SAPIEN supplies articulated objects and physics-based manipulation. iGibson 1.0 combines interactive scenes with demonstration tools, and iGibson 2.0 adds object-state changes and a VR interface [158,259,342]. BEHAVIOR and BEHAVIOR-1K organize such interactions into household goals and activities [159,281]. An evaluation can vary appearance, layout, object properties, or available actions while retaining a defined task outcome.
Recorded activity and transfer. Ego4D and Ego-Exo4D document everyday and skilled actions from first-person or paired first- and third-person views [79,80]. RoboTHOR studies navigation across simulated and physical environments, while ManiSkill2 evaluates manipulation across objects and task families [51,83]. Recorded actions document what occurred; testing an unrecorded intervention additionally requires an executable environment or a separate experimental protocol.
0pt0pt6pt10pt

4.2. Evaluation Targets, Metrics, and Evidence

Geometry, perception, and operation. Text2Room’s judgments concern perceived scene structure; TRELLIS.2’s reconstruction tests concern reference geometry [103,345]. CoGen3D’s imported-object manipulation illustrates evaluation at the system level [132].
Camera motion and scene dynamics. Spatial evaluation separates viewpoint change from change within the world. WorldScore measures camera adherence, static-world agreement estimated from DROID-SLAM depth and co-visible-pixel reprojection, and requested in-scene motion under a fixed camera [61].
Validation of model-based judgments. Automated judgments require calibration against the endpoint they intend to measure. 4DWorldBench assesses perceptual quality, condition alignment, physical realism, and 4D consistency under text, image, and video conditions. It uses model-based and language/vision-language-model judgments [193]. Against ten participants’ ratings of 100 videos, its initial style-consistency ablation increases Pearson correlation from 0.383 to 0.545; Spearman correlation remains 0.457. Linear association with human ratings improves, while rank agreement is unchanged.
Stereoscopic viewing experience. Perceptual evaluation depends on both generated content and its display conditions. The Stereoscopic Content Preference Evaluation (SCOPE) dataset records static-stereo preferences on Apple Vision Pro; iSQoE predicts quality of experience from these annotations [291]. A separate comparison uses Apple Vision Pro, Meta Quest Pro, anaglyph viewing, and alternating monocular monitor views. These conditions distinguish geometric proxies, desktop preferences, and headset judgments. The resulting evidence concerns static-stereo viewing.
Feature-based and distributional measures. CLIP aligns image and text representations [236]; TRELLIS compares rendered views with an input image, whereas Text2Room compares them with text [103,344]. LPIPS approximates image perceptual similarity using calibrated deep features [392]; SSIM compares structure, luminance, and contrast under its signal model [320]. Semantic correspondence can therefore improve while less of a reference view is preserved.
Fréchet Inception Distance compares fitted feature distributions; FVD extends this assessment to video features [113,114,115,116] [97,300]. Because Inception-v3 and DINOv2 represent different learned visual spaces [217,288], Table 3 retains separate Inception, DINO, and point-feature distances. VBench characterizes generated video across multiple quality dimensions [125].
Measures should follow the user-facing claim. Presence questionnaires describe subjective experience [256,332]. Schwind et al. found unchanged mean presence scores for questionnaires administered inside versus outside VR, although response variance and interruptions differed [257]. Studies should therefore record the rating procedure, preceding task, and display conditions. The Simulator Sickness Questionnaire measures symptoms; a review of head-mounted VR studies associates sickness with participant, content, and exposure factors [141,252]. Presence ratings should accordingly be accompanied by exposure duration, withdrawals, and discomfort.
Workload, usability, and successful use. The NASA Task Load Index measures perceived workload dimensions; IBM usability questionnaires assess satisfaction with system usability [93,156]. These ratings and time savings should accompany task criteria, including correction accuracy, multimodal reference resolution, or coordinated manipulation. Task criteria distinguish completion of the intended operation from acceptance of an incorrect result.
Reading the evidence profiles. The categories describe evidence types and do not rank capabilities. E denotes a measured result for a named property and setup, including mixed or negative findings; D denotes a demonstrated implemented function. C denotes a source claim without a located targeted test, and I denotes a mechanism-based inference. NR means no evidence was located in the inspected source, not that the capability is absent; NA denotes a property outside the selected output’s scope. Temporal entries distinguish evolution, ordered views, revisiting, and retained modifications. For operating cost, E requires a comparison, profile, configuration experiment, or repeated/averaged measurement; an isolated approximate timing remains C.
Figure 10. Evidence profiles for 31 methods and systems, with property, configuration and temporal-target keys. E/ IDs identify companion records described in Appendix A.
Figure 10. Evidence profiles for 31 methods and systems, with property, configuration and temporal-target keys. E/ IDs identify companion records described in Appendix A.
Preprints 240396 g010

4.3. Source-Reported Quantitative Comparisons

Table 3, Table 4 and Table 5 compare assets, scenes, and dynamic worlds under eight source protocols. Each block retains comparison methods, inputs, test sets, metric directions, and reported precision so that trade-offs remain interpretable within the original task.
Assets: appearance and part structure. TRELLIS-L improves appearance- and shape-distribution scores over the listed Toys4k baselines. Across all three datasets, PartCrafter has lower Chamfer distance and inter-part overlap and higher F-score than TripoSG–HoloPart [175,344]. Overlap measures generated-part intersection, rather than segmentation agreement with ground truth.
Scenes: camera adherence and visual quality. Text2Room combines image scores with perceived completeness; Voyager measures fidelity to held-out observations. WorldScore compares complete configurations with different control interfaces. Its displayed 3D methods receive precise camera poses and score 84.60–94.01 on camera control; the displayed video methods lack that input and score 22.39–38.27. The contrast combines representation and control-interface differences. Hailuo leads object control and CogVideoX-I2V leads style consistency. Camera adherence and visual instruction fulfillment therefore remain separate selection targets. Table 4 preserves the benchmark scores and evaluation settings [61,103,123].
Dynamics: reconstruction, return, and shared observations. DreamGaussian4D evaluates novel-view fidelity and offline generation time for prepared dynamic content. WorldPlay’s short-trajectory PSNR is 21.92 dB versus Gen3C’s 21.68 dB; on long revisits, the scores are 18.94 and 15.37 dB. The long protocol references the initial generated pass, so the two protocols assess different reconstruction targets. MultiGen raises opponent-presence accuracy from the split-screen baseline’s 65.31% to 75.38% on one Doom map. Its rollouts demonstrate player removal and reintroduction in shared memory, with opponent presence as the quantitative endpoint [227,243,285]. MultiGen’s baselines include IP-Adapter, which introduces image prompting through separate image and text attention [371]. Section 6.3 adds an edit-and-return test for retained object changes.
Table 3. Reported asset-generation results under two source protocols. A: Single-image input; 1,250 Toys4k objects; one asset/input. FD/KD: Fréchet/kernel distance. FD-point uses PointNet++ features of depth-derived point clouds. CLIP and both KD columns use the source’s × 100 scale.
Table 3. Reported asset-generation results under two source protocols. A: Single-image input; 1,250 Toys4k objects; one asset/input. FD/KD: Fréchet/kernel distance. FD-point uses PointNet++ features of depth-derived point clouds. CLIP and both KD columns use the source’s × 100 scale.
Preprints 240396 i009
Table 4. Scene-generation results under three source protocols. Arrows give preferred directions; bold marks within-block optima.
Table 4. Scene-generation results under three source protocols. Arrows give preferred directions; bold marks within-block optima.
Preprints 240396 i010
Table 5. Dynamic-content and interactive-world performance. Arrows give preferred directions; bold marks within-block optima.
Table 5. Dynamic-content and interactive-world performance. Arrows give preferred directions; bold marks within-block optima.
Preprints 240396 i011

4.4. Cross-Method Qualitative Comparisons

Figure 11 and Figure 12 complement source-reported scores with shared-input comparisons. Both plates retain the source papers’ selected results and comparison conditions.
Assets: appearance and geometric detail.Figure 11 compares six image-conditioned generators on shared excavator and chest inputs. The excavator reveals differences in boom, cabin, and track continuity; the chest reveals differences in spikes, metal trim, and the front ornament. Several outputs retain overall colors while simplifying or distorting these structures; TRELLIS preserves more local detail in these examples. Alongside Table 3, the views show why appearance agreement and geometric completeness require separate inspection.
Figure 11. Image-conditioned 3D asset generation from shared inputs (top row). Each method retains two views and grey geometry insets. Images from Xiang et al.(2025)Xiang, Lv, Xu, Deng, Wang, Zhang, Chen, Tong, and Yang [344].
Figure 11. Image-conditioned 3D asset generation from shared inputs (top row). Each method retains two views and grey geometry insets. Images from Xiang et al.(2025)Xiang, Lv, Xu, Deng, Wang, Zhang, Chen, Tong, and Yang [344].
Preprints 240396 g011
Scenes: content retention under a viewpoint change.Figure 12 compares Voyager, FlexWorld, SEVA, See3D, and ViewCrafter on four shared examples. The staircase reveals displaced surfaces and distorted furniture; the bathroom reveals lost or altered small objects. Alongside Table 4, they show that plausible room appearance need not preserve the reference scene’s structure or details. Methods retain their native conditioning representations, and the displayed target frames support single-view comparisons [123].
Figure 12. Camera-controlled scene generation across four examples. Rows show the starting input, observed target-view reference, and five methods at corresponding target views. Red boxes mark selected discrepancies. Images from Huang et al.(2025c)Huang, Zheng, Wang, Liu, Wang, Wu, Jiang, Li, Lau, Zuo, and Guo [123].
Figure 12. Camera-controlled scene generation across four examples. Rows show the starting input, observed target-view reference, and five methods at corresponding target views. Red boxes mark selected discrepancies. Images from Huang et al.(2025c)Huang, Zheng, Wang, Liu, Wang, Wu, Jiang, Li, Lau, Zuo, and Guo [123].
Preprints 240396 g012

4.5. Costs of Generation, Rendering, and Interaction

Timing endpoints.Table 6 separates preparation, component execution, and generation during use by timed stage, output, and hardware. WorldPlay’s progressive decoding targets earlier feedback, whereas its speed table measures throughput [285]. Input-to-display latency also depends on buffering, scheduling, transfer, and presentation.
Component time and complete response. TRELLIS.2’s A100 experiment measures shape-decoder time; its H100 figures describe full generation [345]. Generated Reality times generation and decoding; the display interval is unreported. Section 6.4 specifies complete-response measurement.
Time to usable content. The relevant endpoint is the first state that supports the intended operation. In the heritage study’s three-pair comparison, raw generation takes approximately 3–4 minutes. Basic structural editability requires 4–5 hours of total post-processing, including 1–2 hours of close-up visual refinement [181]. These measurements characterize a small hosted-service preparation workflow.
Table 6. Reported operating costs by stage, hardware, and output setting. Units: gen. FPS generated frames/s; b batch size. Evidence: E timing evaluation; C isolated timing report. Both are source-reported (Section 4.2).
Table 6. Reported operating costs by stage, hardware, and output setting. Units: gen. FPS generated frames/s; b batch size. Evidence: E timing evaluation; C isolated timing report. Both are source-reported (Section 4.2).
Preprints 240396 i012

5. XR Systems, Workflows, and Model Selection

Table 7 links documented XR systems to their models, artifacts, platforms, tasks, and evaluations. We follow these configurations from generation through preparation and interaction to the measured user outcome.

5.1. Documented Model-to-System Links

Assembly and asset-delivery pipelines. Layout generation enters XR through object structure retained by the application. VRCopilot integrates ATISS, which autoregressively predicts furniture categories and bounding boxes from a room and partial layout. Unity instantiates furniture from an existing library and exposes it for manipulation [222,391]. The model supplies selection and arrangement; the library and application supply meshes and operations.
Image-to-mesh systems extend this handoff to geometry generated from user input. MS2Mesh-XR combines sketch and voice input, ControlNet 1.1 image generation, CRM reconstruction, and runtime OBJ import into Unity [299]. ControlNet supplies spatial conditioning to image diffusion [232] [390]; CRM generates orthographic views and reconstructs a textured mesh through a convolutional triplane representation and FlexiCubes [322]. CoGen3D connects FLUX.1 images and Hunyuan3D-2 to textured GLB delivery in Unity [132]. Its desktop co-design workflow, hardware, and VR study document deployment beyond the Hunyuan3D 2.0 model evaluation [395]. Dreamcrafter combines captured Gaussian splats, meshes, and replaceable generative modules, including LGM or GRM [302].
Generated observations in the display loop. Generated Reality specializes Eq. (26) for tracked head and hand inputs, then streams generated observations through Unity to Quest 3 [348]. Relative to camera-conditioned generation, it adds hand parameters H k and projected skeleton latents z k skel . In the source’s conditioning construction, the transformer input is
X k = Patchify ( [ z k , z k skel ] ch ) + E hand ( H k ) + E cam ( P k ) .
Here z k is the video-latent input, [ · ] ch concatenates channels, and P k contains Plücker embeddings of tracked camera rays. Each hand’s H k includes wrist translation/quaternion and 20 joint angles; learned encoders match the patch-token shape. These 3D parameters complement the aligned 2D skeleton branch. Figure 13 shows encoder inputs and visual-task sequences from the deployed 5B model, distinct from the 14B conditioning experiments.
Figure 13. Generated Reality conditioning and visual responses. (a) Camera, hand-parameter, and skeleton encoders, with a HOT3D image [10]. (b) Button pressing, wheel turning, and jar opening in the 5B headset study, compared with a head-pose/text baseline without tracked hands. Images from Xie et al.(2026b)Xie, Sun, Neall, Wu, Cai, and Wetzstein [348].
Figure 13. Generated Reality conditioning and visual responses. (a) Camera, hand-parameter, and skeleton encoders, with a HOT3D image [10]. (b) Button pressing, wheel turning, and jar opening in the 5B headset study, compared with a head-pose/text baseline without tracked hands. Images from Xie et al.(2026b)Xie, Sun, Neall, Wu, Cai, and Wetzstein [348].
Preprints 240396 g013
XR as an authoring interface. Some systems use XR to direct generation for a separate output medium. VideoCraft uses MR spatial-layer editing to steer concept videos viewed elsewhere; its evaluation concerns authoring control [157]. Context-aware AR generation recommends objects from scene images using a vision–language model; Say It, See It compares speech-driven generation pipelines within AR [17,353]. Their findings concern semantic or aesthetic suitability. Gesture2Prop explores joint speech and hand-gesture conditioning for prop creation in an adjunct demonstration [368].

5.2. Where Operational Capabilities Are Introduced

Operational capabilities often enter the pipeline after visual content is generated. After GLB import, CoGen3D sets scale/orientation heuristically and adds manipulation components for resizing and repositioning [132]. Conversion determines whether materials, hierarchy, and physical properties survive delivery. Dreamcrafter associates Gaussian appearance with meshes for interaction and collision; boundary agreement affects selection and contact precision [302]. A grab component supplies rigid manipulation, whereas articulation and deformation require further structure.
Part-level interaction requires both segmented geometry and rules for its use. HOICraft uses vision-language-model assistance to recommend parts and hand–object interactions after segmentation, labels, and motion constraints are supplied [154]. In its 12-designer study, fewer exploration attempts did not yield a statistically significant decision-speed improvement. PartInteractor instead exposes generated part hierarchies for selection, removal, and recombination in XR [131]. This supports component editing, while maintaining semantic part identity during later interaction remains an implementation issue.
Generating behavior through orchestration. LLMR creates and modifies interactive worlds through generated code, while ImoGenXR combines asset retrieval with generated behaviors in immersive authoring [50,169]. Executable behavior comes from orchestration and author control beyond the visual generator’s output. GenAssist retrieves models and documentation to generate and check executable scene programs; its experiments use a browser viewer [280]. CreepyCoCreator studies AI representation and edit presentation through Wizard-of-Oz co-creation. Its object additions and replacements are system-selected and artist-created [241]. Its findings concern interaction and expectations within this simulated co-creation workflow.

5.3. Authoring and Interaction Workflows

Staging decisions before final generation. Authoring systems expose intermediate content so users can guide generation before final delivery. VRCopilot uses reviewable wireframes in manual, scaffolded, and automatic modes [391]. Dreamcrafter records online-preview annotations for broker-executed generation [302]; CoGen3D uses asynchronous jobs [132]. MS2Mesh-XR places image selection before reconstruction and import [299]. These workflows differ in when users commit decisions and how those decisions transfer to the final asset. Section 4.5 distinguishes the corresponding preview, delivery, and display intervals.
Spatial input and editable representations. The representation determines what a spatial gesture can change. GaussianShopVR combines immersive selection and drawing with Gaussian-based editing and object generation; DreamSpace propagates panoramic appearance across captured geometry [262,360]. Generative terrain authoring maps mid-air hand contours to a learned terrain representation [110]. These interfaces expose different editable units: points, surfaces, objects, or scene regions. Local appearance changes, articulation, and nearby object structure consequently require distinct editing support.
Situated creation and physical context. Dream Mesh connects speech-driven generation to mesh import and manipulation in MR [330]. Roomify transforms captured rooms while preserving approximate object footprints, major contact surfaces, and high-level functional roles. Editing Reality explores in-situ speech, sketching, and direct manipulation, informed by a formative workshop and expert review [319,331]. FusionProtor uses component-level physical-to-virtual transitions for mixed prototypes and subsequent simulation [388].
Generation during collaboration. Generated content can be the shared design object or the setting for collaboration. Thing2Reality explores avatar decoration, collaborative furniture layout, and toy ideation/pitching through shared two-dimensional content and three-dimensional objects [108]. Furniture layout emphasizes spatial manipulation, while ideation also uses a two-dimensional whiteboard. SpaceBlender compares blended familiar spaces with Generic3D and Text2Room during collaborative sticky-note clustering, without manipulating generated room surfaces [214]. These studies address the awareness, communication, and coordination concerns of collaborative MR [65].

5.4. VR, AR, and MR Integration Conditions

Alignment with physical space. Tracked input and environmental registration address different problems in XR integration. MS2Mesh-XR supplies drawing, speech, and manipulation interfaces and demonstrates manual resizing and positioning of MR furniture [299]. Its inspected implementation and use-case sections provide no registration-error, real-object occlusion, or lighting-adaptation measurements. ImaginateAR represents pre-scanned surroundings with labeled bounding boxes and a scene graph. A visual positioning system localizes the mobile device, and the graph guides asset placement [155]. Sensing and localization supply registration. Reported evaluations concern components, technical performance, and authoring.
Viewing geometry. The display pipeline determines the spatial queries supported by VR content. Images, panoramic backgrounds, and meshes rendered from two eye cameras produce different depth behavior even when physical-surface registration is unnecessary. Dreamcrafter’s Magic Camera produces a stylized snapshot [302]. Extending such content to head translation requires binocular consistency and disocclusion to be checked in the resulting view-generation pipeline.
EPIC addresses binocular panoramic video through left/right view generation, spherical-epipolar-preference refinement, and omnidirectional-stereo preparation [196]. Before binocular super-resolution, preparation re-renders both eyes from the left view using depth scaled against the right view. Geometric scores assess stereo, temporal, and cross-time cross-eye residuals on rigid-scene correspondences, excluding independently moving tracks and invalid low-parallax pairs. Refinement improves stereo residuals but worsens cross-time cross-eye residuals on this subset.

5.5. Headset Demonstrations and User Tasks

Agency and authoring experience. User studies reveal how integration choices affect control over generated content. VRCopilot reports a 14-participant study and a second with 15 different Quest 2 participants [391]. In the second, bedroom-authoring agency is highest for manual creation, followed by scaffolded and automatic creation. This ordering links authoring structure to perceived control; circulation and window access add practical layout constraints.
Dreamcrafter’s seven-person exploratory study uses computer-linked Quest 3 for holiday scenes [302]. Participants report greater control from sculpting before stylization, uncertain final appearance, and lost size information in image previews. Low-fidelity mesh previews for prompted objects were added after the study. CoGen3D studies 60 design participants and an independently recruited validation group of 60 in affective VR scenes [132]. Satisfaction is higher for generated images than resulting 3D assets.
Visual action response. Generated Reality evaluates tracked-hand tasks through blinded judgments of generated recordings and perceived control [348]. Hand conditioning improves task accuracy over a camera/text baseline lacking tracked hands. The measured endpoint is perceived and judged visual response. MS2Mesh-XR contributes VR/MR demonstrations and stage timings [299].
Application-level presentation. ImmerseGen prepares lightweight textured scenes for headset rendering, while CinemaWorld generates film-linked spatial augmentations before playback [128,383]. ShadAR injects generated shaders into monoscopic passthrough augmentation during use. It measures code-generation delay separately from frame-update throughput [199].
Heritage visualization combines hosted generation, manual inspection, and Unity/PICO 4 presentation [181]. Questionnaires and interviews cover four painting-derived scene fragments, nine experts, and 30 non-experts. Experts contribute either on-site VR or remote first-person video judgments. Topology and preparation effort are assessed separately on three AI/manual object pairs.
Han and Jeong compare static panoramas generated with Skybox AI and image processing against SketchUp/Enscape references, using seven Quest 3 experts [90]. Image-property and presentation-quality judgments provide device-based evidence for static viewing.
Table 7. Catalogue of XR authoring systems and generated-content presentation methods.
Table 7. Catalogue of XR authoring systems and generated-content presentation methods.
Preprints 240396 i013
Preprints 240396 i014

5.6. Task Requirements and Model Selection

Paradigm-level requirements such as efficiency and consistency narrow the choice of generation route [347]. Within a route, selection depends on the exact operation, the state or artifact it changes, the component implementing that change, and the endpoint tested. The following comparisons show how this evidence changes a choice.
Modify a part, preserve contact, and return. Consider moving a chair back independently, maintaining the intended contact/collision behavior, and finding the accepted change after leaving and returning. CoGen3D demonstrates whole-object manipulation through Unity components applied to Hunyuan3D-2 assets [132]. PartCrafter supplies separate meshes; its geometric scores assess decomposition [175]. PartInteractor provides direct XR component-editing evidence; HOICraft configures behavior on prepared parts and constraints [131,154]. These are alternative starting points with different preparation costs. The task requires stable part identity, an implemented update, compatible collision geometry, and stored accepted state. An integrated pipeline would need to combine these components and test the complete sequence; Figure 14 specifies the retention test.
Explore a generated room with tracked head motion. A prepared scene with local rendering exposes camera queries independently of later generation. Text2Room supplies a surface for walkthroughs; Holodeck adds addressable scene objects, and HY-World 2.0 supplies a separate collision proxy [103,298,365]. These structures support geometric queries; binocular viewing requires assessment in the headset configuration. WorldPlay instead generates views from camera/action conditions and retrieved history, with quantitative camera and revisit evidence. Generated Reality supplies a 5B tracked-head/hand deployment and short visual-task evaluation [285,348]. These routes permit generated visual responses, with reported timings for generation throughput or decoding delay. The choice trades explicit queryable geometry and local viewpoint updates against ongoing synthesis. Selection depends on supported view changes, disocclusion, stereo behavior, and the complete response path.
What changes the choice. Rough composition can use a preview before geometry is complete; precise placement depends on the final boundary and scale, as Dreamcrafter’s findings illustrate (Section 5.5). An edit-and-return requirement instead makes the supported update and retained variable decisive. WorldMem and PERSIST address changed content within their respective settings [77,346]; those settings must match the requested edit before their persistence evidence transfers. Table 6 then constrains when the required output becomes available.

6. Challenges and Future Directions

We propose five matched evaluation designs for transferring model-level evidence to XR operations. Each holds a relevant task or content reference fixed while varying a stage in the pipeline.

6.1. Binocular Consistency Under Head Motion

Transferring stereo consistency from prescribed views to tracked head motion requires evaluating newly requested binocular views. StereoDiffusion generates stereo image pairs using diffusion, and StereoCrafter converts monocular video to stereoscopic video [314,394]. StereoWorld adds geometry-aware monocular-to-stereo generation; Stereo World Model targets camera-guided stereo synthesis [286,351]. This transfer adds camera control and response timing to fixed-pair and stereo-sequence evaluation.
A paired study could test whether offline consistency predicts binocular viewing quality for the same content. The study would link findings from cross-view, static-stereo, and device-based panorama evaluations [90,291,297]. Link each asset to its displayed version and record viewing queries, rendering configuration, stereo baseline, depth scale, display settings, and correction/reprojection. Use separate applicable conditions for fixed stereo pairs, panoramic rotation, and translated-view generation. Include any reconstruction or renderer used for translation in the pipeline evaluation.
Within each condition, compare geometric errors and artifacts with task performance and participant judgments. Retain disagreements, such as improved appearance with unchanged binocular error, when interpreting the results. Vary rendering and display settings to separate representation from presentation effects, and validate dynamic content separately.

6.2. Consistency Between Previews and Final Assets

In preview-based authoring, users accept size, placement, and appearance decisions before final geometry becomes available. Dreamcrafter’s selected preview images can condition subsequent 3D generation [302]. Evaluation should determine which decisions survive replacement and how the interface or generator can preserve them.
Retain the preview, user operations, and final asset for each request. A presentation-only experiment could replay requests with the same final asset and matched delivery times, counterbalancing preview format and order. A complementary live experiment could randomize preview interfaces across comparable requests while fixing the task and generator configuration. Preserve selected-preview conditioning where used, and record the assets generated from those choices. Replay isolates displayed-proxy effects, whereas live evaluation tests the complete preview-to-final workflow.
Compare three-dimensional proxies through occupied volumes, relative scale, contact regions, and boundaries in common coordinates. For two-dimensional previews, use projected outlines and the placement constraints they convey. Measure constraint satisfaction, revised decisions, and total correction time after replacement. The generator and authoring system take on different responsibilities when preserving bounding structure, transferring edits, or delaying geometry-dependent operations.

6.3. Persistent and Shared Object State

Persistent interaction requires a successful update to survive absence and remain consistent across observers. Different variables are available in WorldPlay’s visual history, MultiGen’s shared player state, and Holodeck/HY-World 2.0 simulation structures [227,285,298,365]. MultiGen’s level map remains fixed during rollout, limiting the updates that can be tested. LiveWorld evaluates off-screen events using scripts and VLM judgments. Programmable World Model checks engine-driven entity counts and death events, although event success is not tied to a specified entity [62,126]. An XR test should identify the supported action, changed variable, and storage component. Choose an object edit, a supported transition, or an unchanged-scene revisit according to the interface.
For a supported edit, modify an object, move until it is out of view, and return through a different route. Report update success before departure, retention among successful updates, and overall completion separately. A matched unmodified revisit provides the reference for testing visual recall. Assess identity, pose, appearance, and action-relevant state separately: a visually closed door need not have closed collision or traversal state. Test the action’s expected visible consequence, and compare engine state with generated observations when that state is accessible.
Figure 14 pairs the same scene and return viewpoint with references to the earlier observation or accepted modification.
Figure 14. Proposed recall and retention tests on matched C0–C1–C0 routes, with the chair hidden at C1. Return references are (a) original position A and (b) accepted edited position B. Update success and retention are scored separately. All scenes are schematic.
Figure 14. Proposed recall and retention tests on matched C0–C1–C0 routes, with the chair hidden at C1. Return references are (a) original position A and (b) accepted edited position B. Update success and retention are scored separately. All scenes are schematic.
Preprints 240396 g014
Vary the return interval, route, occlusion, and repeated operations. Trace failures to action interpretation, storage, retrieval, or rendering to identify which retained information the task needs.
A collaborative test would inspect the same accepted update from multiple viewpoints before introducing delayed or conflicting inputs. MultiGen’s opponent-presence evaluation and Solaris’s two-player Minecraft tasks already test cross-observer behavior [227,253]. For a specified update, compare stored state, generated observations, and outcomes. This would distinguish disagreement about the update from a mismatch in its presentation.
Control memory budget and history quality independently (Section 3.3.5); test extended absence separately from short action response.

6.4. Response Latency and System Scheduling

Response scheduling connects model execution to the time when a user sees a useful change. Generated Reality reports generation/decoding delay [348]. A complete response trace would also timestamp input receipt, queuing/scheduling, memory retrieval, transfer, rendering, and display. Record applicable loading/import stages, dependencies, and overlap alongside the generation stage.
With display and task fixed, vary input frequency, scene complexity, hardware allocation, and network conditions. Measure time to the first meaningful response and to a usable final result. Also record displayed-content age and the frequency of interrupted or superseded requests. Report both average and tail delays, tracing head-motion updates separately when a local scene keeps rendering during generation.
These traces support task-specific comparisons of synchronous generation, preview-based authoring, and local rendering with asynchronous updates. A scheduling ablation should fix the input trace, model/output settings, hardware, and network while varying scheduling or presentation. Report visual quality and task completion alongside latency, since provisional responses may support fewer operations than completed results.

6.5. Physical-Space Alignment and Adaptation

Physical-space adaptation requires generation, placement, and display to preserve constraints from supporting surfaces, room boundaries, foreground occluders, and illumination. ImaginateAR supplies context through a pre-scanned graph and localization pipeline [155]. The question is how well each constraint survives across these stages and after the physical context changes.
A controlled task could generate and place a constrained asset, then revisit it after viewpoint or environmental changes. Measure support-relative placement error, reference-relative scale error, occlusion-order violations, and task success. Record whether spatial inputs come from manual specification, sensing, or inference. Fixing the asset while varying alignment can help isolate registration effects. Fixing registration while varying the asset tests preservation of the required geometry.
Vary illumination and partial occlusion to test adaptation to environmental changes [118,172,173,174,195]. Evaluate geometric fit, appearance adaptation, and interaction outcomes separately to locate failures in generation, alignment, or presentation.

7. Conclusion

This review connects visual generative models to XR use by following the path from generated output to a tested operation. Common mathematical descriptions explain how conditioning inputs, representations, and update mechanisms differ across assets, scenes, and dynamic worlds. The evidence profiles and deployment traces then identify the components and configurations supporting the reported capabilities.
The comparisons reveal a recurring distinction between producing content and maintaining the structures needed to operate on it. Separate part meshes expose a possible editing unit; manipulation and retained edits additionally depend on the update and storage components supplied by the complete configuration. Prepared scenes permit local viewpoint queries, whereas observation generators synthesize new views from actions and history. These routes place different demands on geometry, memory, and response scheduling. Accordingly, selection depends on the editable unit, the state that must survive, and the time at which a useful response becomes visible.
The five proposed matched evaluations are designed to test how model-level evidence transfers through viewing, authoring, state updates, scheduling, and physical-space adaptation. They separate successful updates from later retention and component speed from the response experienced by a user. Progress should therefore be assessed through the viewing, editing, and interaction tasks supported by a complete XR configuration.

Supplementary Materials

The following supporting information can be downloaded at the website of this paper posted on Preprints.org.

Author Contributions

G.H. was responsible for the conception and design of the review, literature curation and synthesis, development of the analytical framework, preparation of figures and tables, and drafting and revision of the manuscript. G.H. takes responsibility for the manuscript’s content.

Funding

No funding was received for this work.

Data Availability Statement

The review-level data, including catalogue metadata, XR capability annotations, source-linked extraction records, and source-reported quantitative comparison data, are provided in the accompanying Evidence supplement E. The underlying publications are cited in the manuscript and linked in the supplementary records.

Conflicts of Interest

The author declares no conflicts of interest.

Use of Artificial Intelligence

OpenAI’s ChatGPT and Codex tools assisted literature retrieval and source inspection, evidence organization and comparison, manuscript drafting and revision, figure and table preparation, and code generation for document preparation and LaTeX compilation checks. The author takes responsibility for the accuracy, interpretation, citations, and final content of the manuscript.

Appendix A. Review Records and Evaluation Details

Scope and selection. This is a critical narrative review based on purposive retrieval, primary-source checks, and citation-led follow-up. Selection sought complementary generation mechanisms, output structures, and documented XR operations. Foundational reconstruction, rendering, and learning papers supply technical context; they are distinguished from the 204 catalogued methods/systems and 37 resources. The final update used targeted retrieval through 23 October 2026. Search-result and full-exclusion counts were not retained.
Detailed evidence subset. The 31 coded methods and systems form a deliberately selected explanatory subset spanning assets, scenes, dynamics, and XR integration. Eight dimensions yield 248 evidence cells; three benchmark records and nine neighboring-survey records provide context but no additional cells. Coding records the named property, inspected source, source location, and qualification associated with each code. Source material includes technical papers, selected official implementation files, and the Genie 3 official description. Static code inspection documents export paths; execution and device use require separate runtime evidence. Codes were developed through author-side, AI-assisted extraction and source checks. Independent duplicate human screening and intercoder agreement are not documented. Later catalogue additions, including WorldMem, PERSIST, and Solaris, inform the task synthesis and agenda outside this coded subset. The map supports case comparisons; field-wide capability prevalence lies outside this purposive sample’s scope.
Traceability and interpretation. Evidence supplement E includes a browsable record index and a catalogue crosswalk. Its E/A01-style IDs distinguish evidence records from catalogue IDs. The accompanying evidence-matrix.csv links each cell to its note, source, and location; extraction-records.json retains source versions and inspected configurations. search-log.txt records the retained queries and follow-ups, and the quantitative evidence files record source protocols and result strings. Codes describe evidence for a specified property and configuration; comparisons therefore use the associated cell notes. Quantitative blocks retain their own inputs, references, workloads, and precision, with no pooled ranking. Technical editions are recorded at the point of extraction: Table 3A uses TRELLIS arXiv v1, whereas WorldPlay’s coded profile uses arXiv v2 and Table 5 uses its PMLR edition.
Table A1 lists the experimental subsets and workloads underlying the quantitative comparisons in Section 4.3.
Table A1. Data resources and workloads used in the quantitative comparisons.
Table A1. Data resources and workloads used in the quantitative comparisons.
T: training resource; E: evaluation context. Counts reflect paper-specific curation; evaluation rows correspond to Tables 3–5.
ID Dataset / context Role Scale Unit Used by / source
Training resources
T1 TRELLIS curated union Asset training 500,777 3D objects TRELLIS [344]
T2 Objaverse / ShapeNet / ABO, part-labeled Part training ≈50,000 / ≈300,000 objects / parts PartCrafter [175]
T3 RealEstate / DL3DV / UE mixture Video training >100,000 curated clips Voyager [123]
T4 Sekai / DL3DV / UE / game mixture World training ≈320,000 30–40 s clips WorldPlay [285]
Evaluation protocols
E1 Toys4k subset Image-to-3D 1,250 test objects TRELLIS [344]
E2 Objaverse / ShapeNet / ABOa Part generation ≈2,000a overall test samples PartCrafter [175]
E3 Text2Room room promptsb Room quality 60 / 61 views per room / raters Text2Room [103]
E4 RealEstate10K test subset Novel views 150 test clips Voyager [123]
E5 WorldScore World evaluation 3,000 specificationsc WorldScore [61]
E6 Consistent4D subsetd Video-to-4D 7 × 32 videos × frames DG4D [243]
E7 DL3DV / game / AI-image test suite Control / revisit 600 test cases WorldPlay [285]
E8 ViZDoom multiplayer contexte Opponent presence 1 / 2 map / test players MultiGen [227]
aAbout 2K overall; per-dataset counts and object/scene allocation unspecified. T2 excludes the separate 30K monolithic regularization pool. bRoom count NR; 61 raters, 1,098 datapoints. c2,000 static and 1,000 dynamic specifications; Table 4(c) uses static scores. dFour novel evaluation views/video; DG4D-Fast uses 16 input frames. eOne map, two test players; test-trajectory count NR. The >10-million-frame training collection is separate. T4 counts curated clips; E7 source-specific test counts are NR.

References

  1. Achlioptas, P.; Diamanti, O.; Mitliagkas, I.; Guibas, L. Learning Representations and Generative Models for 3D Point Clouds; ICML, 2018; pp. 40–49. [Google Scholar]
  2. Akeley, K.; Watt, S.J.; Girshick, A.R.; Banks, M.S. A stereo display prototype with multiple focal distances. ACM Trans. Graph 2004, 23(3), 804–813. [Google Scholar]
  3. Alonso, E.; Jelley, A.; Micheli, V.; Kanervisto, A.; Storkey, A.; Pearce, T.; Fleuret, F. Diffusion for World Modeling: Visual Details Matter in Atari; NeurIPS, 2024. [Google Scholar]
  4. Armeni, I.; Sener, O.; Zamir, A.R.; Jiang, H.; Brilakis, I.; Fischer, M.; Savarese, S. 3D Semantic Parsing of Large-Scale Indoor Spaces; CVPR, 2016; pp. 1534–1543. [Google Scholar]
  5. Assran, M.; Bardes, A.; Fan, D.; Garrido, Q.; Howes, R.; Komeili, M.; Muckley, M.; Rizvi, A.; Roberts, C.; Sinha, K.; Zholus, A.; Arnaud, S.; Gejji, A.; Martin, A.; Hogan, F.R.; Dugas, D.; Bojanowski, P.; Khalidov, V.; Labatut, P.; Massa, F.; Szafraniec, M.; Krishnakumar, K.; Li, Y.; Ma, X.; Chandar, S.; Meier, F.; LeCun, Y.; Rabbat, M.; Ballas, N. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv 2025, arXiv:2506.09985. [Google Scholar]
  6. Azuma, R.; Bishop, G. Improving static and dynamic registration in an optical see-through HMD; SIGGRAPH, 1994; pp. 197–204. [Google Scholar]
  7. Azuma, R.; Baillot, Y.; Behringer, R.; Feiner, S.; Julier, S.; MacIntyre, B. Recent advances in augmented reality. IEEE Comput Graph Appl. 2001, 21(6), 34–47. [Google Scholar]
  8. Azuma, R.T. A Survey of Augmented Reality. Presence (Camb) 1997, 6(4), 355–385. [Google Scholar]
  9. Bahmani, S.; Skorokhodov, I.; Rong, V.; Wetzstein, G.; Guibas, L.; Wonka, P.; Tulyakov, S.; Park, J.J.; Tagliasacchi, A.; Lindell, D.B. 4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling; CVPR, 2024; pp. 7996–8006. [Google Scholar]
  10. Banerjee, P.; Shkodrani, S.; Moulon, P.; Hampali, S.; Han, S.; Zhang, F.; Zhang, L.; Fountain, J.; Miller, E.; Basol, S.; Newcombe, R.; Wang, R.; Engel, J.J.; Hodan, T. HOT3D: Hand and object tracking in 3D from egocentric multi-view videos; CVPR, 2025; pp. 7061–7071. [Google Scholar]
  11. Banks, M.S.; Read, J.C.A.; Allison, R.S.; Watt, S.J. Stereoscopy and the Human Visual System. SMPTE Motion Imaging J. 2012, 121(4), 24–43. [Google Scholar]
  12. Bar-Tal, O.; Chefer, H.; Tov, O.; Herrmann, C.; Paiss, R.; Zada, S.; Ephrat, A.; Hur, J.; Liu, G.; Raj, A.; Li, Y.; Rubinstein, M.; Michaeli, T.; Wang, O.; Sun, D.; Dekel, T.; Mosseri, I. Lumiere: A Space-Time Diffusion Model for Video Generation. SIGGRAPH Asia Conference Papers, 2024; pp. 1–11. [Google Scholar]
  13. Barron, J.T.; Mildenhall, B.; Tancik, M.; Hedman, P.; Martin-Brualla, R.; Srinivasan, P.P. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. ICCV 2021. [Google Scholar]
  14. Barron, J.T.; Mildenhall, B.; Verbin, D.; Srinivasan, P.P.; Hedman, P. Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. CVPR 2022. [Google Scholar]
  15. Barron, J.T.; Mildenhall, B.; Verbin, D.; Srinivasan, P.P.; Hedman, P. Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields; ICCV, 2023. [Google Scholar]
  16. Baruch, G.; Chen, Z.; Dehghan, A.; Dimry, T.; Feigin, Y.; Fu, P.; Gebauer, T.; Joffe, B.; Kurz, D.; Schwartz, A.; Shulman, E. ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data. NeurIPS Datasets Benchmarks Track 2021. [Google Scholar] [CrossRef]
  17. Behravan, M.; Matković, K.; Gračanin, D. Generative AI for Context-Aware 3D Object Creation Using Vision-Language Models in Augmented Reality. IEEE AIxVR, 2025. [Google Scholar]
  18. Billinghurst, M.; Kato, H.; Poupyrev, I. The MagicBook: a transitional AR interface. Comput Graph 2001, 25(5), 745–753. [Google Scholar]
  19. Blattmann, A.; Dockhorn, T.; Kulal, S.; Mendelevitch, D.; Kilian, M.; Lorenz, D.; Levi, Y.; English, Z.; Voleti, V.; Letts, A.; Jampani, V.; Rombach, R. Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets. arXiv 2023a, arXiv:2311.15127. [Google Scholar]
  20. Blattmann, A.; Rombach, R.; Ling, H.; Dockhorn, T.; Kim, S.W.; Fidler, S.; Kreis, K. Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models. CVPR 2023b. [Google Scholar]
  21. Bokhovkin, A.; Meng, Q.; Tulsiani, S.; Dai, A. SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generation. CVPR 2025. [Google Scholar]
  22. Bowman, D.A.; Hodges, L.F. An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments; SI3D, 1997; pp. 35–ff. [Google Scholar]
  23. Bruce, J.; Dennis, M.D.; Edwards, A.; Parker-Holder, J.; Shi, Y.; Hughes, E.; Lai, M.; Mavalankar, A.; Steigerwald, R.; Apps, C.; Aytar, Y.; Bechtle, S.M.E.; Behbahani, F.; Chan, S.C.; Heess, N.; Gonzalez, L.; Osindero, S.; Ozair, S.; Reed, S.; Zhang, J.; Zolna, K.; Clune, J.; Freitas, N.D.; Singh, S.; Rocktäschel, T. Genie: Generative Interactive Environments; ICML, 2024. [Google Scholar]
  24. Caldeira, H.; Wang, H.; Huang, G.; Cai, S.; Fu, R.; Anantrasirichai, N. 2026) Geometry beneath the Waves: Dense Priors for Sparse-View Underwater 3D Gaussian Splatting. SIGGRAPH Asia 2026 Posters;arXiv v2, accepted; technical version inspected; 2026, arXiv:2609.18737v2. [Google Scholar]
  25. Campos C, Elvira R, Rodriguez JJG, M. Montiel JM, D. Tardos J ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM. IEEE Trans. Robot 2021, 37(6), 1874–1890.
  26. Chan, E.R.; Lin, C.Z.; Chan, M.A.; Nagano, K.; Pan, B.; Mello, S.D.; Gallo, O.; Guibas, L.J.; Tremblay, J.; Khamis, S.; Karras, T.; Wetzstein, G. Efficient Geometry-Aware 3D Generative Adversarial Networks. CVPR 2022. [Google Scholar]
  27. Chang, A.; Dai, A.; Funkhouser, T.; Halber, M.; Nießner, M.; Savva, M.; Song, S.; Zeng, A.; Zhang, Y. Matterport3D: Learning from RGB-D Data in Indoor Environments; 3DV, 2017. [Google Scholar]
  28. Chang, A.X.; Funkhouser, T.; Guibas, L.; Hanrahan, P.; Huang, Q.; Li, Z.; Savarese, S.; Savva, M.; Song, S.; Su, H.; Xiao, J.; Yi, L.; Yu, F. ShapeNet: An Information-Rich 3D Model Repository. ArXiv 2015, arXiv:1512.03012. [Google Scholar]
  29. Chang, H.; Zhang, H.; Jiang, L.; Liu, C.; Freeman, W.T. MaskGIT: Masked Generative Image Transformer; CVPR, 2022; pp. 11315–11325. [Google Scholar]
  30. Chen, A.; Xu, Z.; Geiger, A.; Yu, J.; Su, H. TensoRF: Tensorial Radiance Fields; ECCV, 2022. [Google Scholar]
  31. Chen, B.; Martí Monsó, D.; Du, Y.; Simchowitz, M.; Tedrake, R.; Sitzmann, V. Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion. NeurIPS 2024a, arXiv:2407.01392. [Google Scholar] [CrossRef]
  32. Chen, D.Z.; Siddiqui, Y.; Lee, H.Y.; Tulyakov, S.; Nießner, M. Text2Tex: Text-driven Texture Synthesis via Diffusion Models; ICCV, 2023a. [Google Scholar]
  33. Chen, D.Z.; Li, H.; Lee, H.Y.; Tulyakov, S.; Nießner, M. SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors. CVPR 2024b. [Google Scholar]
  34. Chen, H.; Zhang, Y.; Cun, X.; Xia, M.; Wang, X.; Weng, C.; Shan, Y. VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models; CVPR, 2024c; pp. 7310–7320. [Google Scholar]
  35. Chen, L.; Zhou, Z.; Zhao, M.; Wang, Y.; Zhang, G.; Huang, W.; Sun, H.; Wen, J.R.; Li, C. FlexWorld: Progressively Expanding 3D Scenes for Flexible-View Exploration. NeurIPS 2025a. [Google Scholar]
  36. Chen, R.; Chen, Y.; Jiao, N.; Jia, K. Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation; ICCV, 2023b. [Google Scholar]
  37. Chen, T.; Hao, K.; Chen, Y.; Cheng, Z.; Xie, R.; Song, L.; Huang, H.; Zhang, C.; Li, X. Full-4D: Generating Full-Scope 4D Scenes from a Single-View Video. arXiv 2026a. [Google Scholar]
  38. Chen, Y.; Chen, Z.; Zhang, C.; Wang, F.; Yang, X.; Wang, Y.; Cai, Z.; Yang, L.; Liu, H.; Lin, G. GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting. CVPR 2024d. [Google Scholar]
  39. Chen, Y.; He, T.; Huang, D.; Ye, W.; Chen, S.; Tang, J.; Cai, Z.; Yang, L.; Yu, G.; Lin, G.; Zhang, C. MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers. ICLR 2025b. [Google Scholar] [CrossRef]
  40. Chen, Z.; Wang, G.; Liu, Z. SceneDreamer: Unbounded 3D Scene Generation From 2D Image Collections. IEEE Trans. Pattern Anal. Mach. Intell. 2023c, 45(12), 15562–15576. [Google Scholar]
  41. Chen, Z.; Tang, J.; Dong, Y.; Cao, Z.; Hong, F.; Lan, Y.; Wang, T.; Xie, H.; Wu, T.; Saito, S.; Pan, L.; Lin, D.; Liu, Z. 3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion. CVPR 2025c. [Google Scholar]
  42. Chen, Z.; Wang, L.; Shen, G.; Yan, D.; Yang, S.; Xu, T.; Du, Y.; Wang, W.; Gui, T.; Huang, L.; Chen, Y. ReWorld: An Interactive World Model with Long-Horizon Memory. ArXiv 2026b, arXiv:2608.23565. [Google Scholar]
  43. Cheng, Y.C.; Lee, H.Y.; Tulyakov, S.; Schwing, A.G.; Gui, L.Y. SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation; CVPR, 2023. [Google Scholar]
  44. Chung, J.; Lee, S.; Nam, H.; Lee, J.; Lee, K.M. LucidDreamer: Domain-Free Generation of 3D Gaussian Splatting Scenes. IEEE Trans. Vis. Comput Graph 2025, 31(12). [Google Scholar]
  45. Collins, J.; Goel, S.; Deng, K.; Luthra, A.; Xu, L.; Gundogdu, E.; Zhang, X.; Yago Vicente, T.F.; Dideriksen, T.; Arora, H.; Guillaumin, M.; Malik, J. ABO: Dataset and Benchmarks for Real-World 3D Object Understanding. CVPR 2022. [Google Scholar]
  46. Cook, R.L.; Torrance, K.E. A Reflectance Model for Computer Graphics. ACM Trans. Graph 1982, 1(1), 7–24. [Google Scholar]
  47. Cummings, J.J.; Bailenson, J.N. How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence. Media Psychol. 2016, 19(2), 272–309. [Google Scholar]
  48. Curless, B.; Levoy, M. A volumetric method for building complex models from range images; SIGGRAPH, 1996; pp. 303–312. [Google Scholar]
  49. Dai, A.; Chang, A.X.; Savva, M.; Halber, M.; Funkhouser, T.; Niessner, M. ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes; CVPR, 2017; pp. 5828–5839. [Google Scholar]
  50. De La Torre, F.; Fang, C.M.; Huang, H.; Banburski-Fahey, A.; Fernandez, J.A.; Lanier, J. LLMR: Real-time Prompting of Interactive Worlds using Large Language Models; CHI, 2024. [Google Scholar]
  51. Deitke, M.; Han, W.; Herrasti, A.; Kembhavi, A.; Kolve, E.; Mottaghi, R.; Salvador, J.; Schwenk, D.; VanderBilt, E.; Wallingford, M.; Weihs, L.; Yatskar, M.; Farhadi, A. RoboTHOR: An Open Simulation-to-Real Embodied AI Platform; CVPR, 2020; pp. 3164–3174. [Google Scholar]
  52. Deitke, M.; VanderBilt, E.; Herrasti, A.; Weihs, L.; Ehsani, K.; Salvador, J.; Han, W.; Kolve, E.; Kembhavi, A.; Mottaghi, R. ProcTHOR: Large-Scale Embodied AI Using Procedural Generation; NeurIPS, 2022; pp. 5982–5994. [Google Scholar]
  53. Deitke, M.; Liu, R.; Wallingford, M.; Ngo, H.; Michel, O.; Kusupati, A.; Fan, A.; Laforte, C.; Voleti, V.; Gadre, S.Y.; VanderBilt, E.; Kembhavi, A.; Vondrick, C.; Gkioxari, G.; Ehsani, K.; Schmidt, L.; Farhadi, A. Objaverse-XL: A Universe of 10M+ 3D Objects. In NeurIPS Datasets and Benchmarks Track; 2023a. [Google Scholar]
  54. Deitke, M.; Schwenk, D.; Salvador, J.; Weihs, L.; Michel, O.; VanderBilt, E.; Schmidt, L.; Ehsani, K.; Kembhavi, A.; Farhadi, A. Objaverse: A Universe of Annotated 3D Objects; CVPR, 2023b. [Google Scholar]
  55. Deng, H.; Pan, T.; Diao, H.; Luo, Z.; Cui, Y.; Lu, H.; Shan, S.; Qi, Y.; Wang, X. Autoregressive Video Generation without Vector Quantization. ICLR 2025, arXiv:2412.14169. [Google Scholar] [CrossRef]
  56. Dhamo, H.; Manhardt, F.; Navab, N.; Tombari, F. Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs; ICCV, 2021. [Google Scholar]
  57. Dhariwal, P.; Nichol, A. Diffusion Models Beat GANs on Image Synthesis; NeurIPS, 2021; pp. 8780–8794. [Google Scholar]
  58. Ding, L.; Dong, S.; Li, Y.; Gao, C.; Chen, X.; Han, R.; Kuang, Y.; Zhang, H.; Huang, B.; Huang, Z.; Wang, Z.; Xu, D.; Xue, T. FullPart: Generating each 3D Part at Full Resolution. ICLR 2026. [Google Scholar] [CrossRef]
  59. Dinh, L.; Sohl-Dickstein, J.; Bengio, S. Density estimation using Real NVP; ICLR, 2017. [Google Scholar]
  60. Downs, L.; Francis, A.; Koenig, N.; Kinman, B.; Hickman, R.; Reymann, K.; McHugh, T.B.; Vanhoucke, V. Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items. IEEE ICRA, 2022. [Google Scholar]
  61. Duan, H.; Yu, H.X.; Chen, S.; Fei-Fei, L.; Wu, J. WorldScore: A Unified Evaluation Benchmark for World Generation; ICCV, 2025; pp. 27713–27724. [Google Scholar]
  62. Duan, Z.; Xia, J.; Zhang, Z.; Zhang, W.; Zhou, G.; Gou, C.; He, Y.; Chen, F.; Zhang, X.; Liu, L. LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models. arXiv 2026. [Google Scholar]
  63. Engstler, P.; Shtedritski, A.; Laina, I.; Rupprecht, C.; Vedaldi, A. SynCity: Training-Free Generation of 3D Worlds. ICCV 2025a. [Google Scholar]
  64. Engstler, P.; Vedaldi, A.; Laina, I.; Rupprecht, C. Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpainting; 3DV, 2025b; pp. 457–468. [Google Scholar]
  65. Ens, B.; Lanir, J.; Tang, A.; Bateman, S.; Lee, G.; Piumsomboon, T.; Billinghurst, M. Revisiting collaboration through mixed reality: The evolution of groupware. Int. J. Hum. Comput Stud. 2019, 131, 81–98. [Google Scholar]
  66. Esser, P.; Rombach, R.; Ommer, B. Taming Transformers for High-Resolution Image Synthesis; CVPR, 2021; pp. 12873–12883. [Google Scholar]
  67. Esser, P.; Kulal, S.; Blattmann, A.; Entezari, R.; Müller, J.; Saini, H.; Levi, Y.; Lorenz, D.; Sauer, A.; Boesel, F.; Podell, D.; Dockhorn, T.; English, Z.; Lacey, K.; Goodwin, A.; Marek, Y.; Rombach, R. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis; ICML, 2024; pp. 12606–12633. [Google Scholar]
  68. Fang, C.; Dong, Y.; Luo, K.; Hu, X.; Shrestha, R.; Tan, P. Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints; 3DV, 2025; pp. 692–701. [Google Scholar]
  69. Feng, W.; Zhu, W.; Fu, T.J.; Jampani, V.; Akula, A.; He, X.; Basu, S.; Wang, X.E.; Wang, W.Y. LayoutGPT: Compositional Visual Planning and Generation with Large Language Models; NeurIPS, 2023. [Google Scholar]
  70. Fisher, M.; Ritchie, D.; Savva, M.; Funkhouser, T.; Hanrahan, P. Example-based synthesis of 3D object arrangements. ACM Trans. Graph 2012, 31(6), 1–11. [Google Scholar]
  71. Fridman, R.; Abecasis, A.; Kasten, Y.; Dekel, T. SceneScape: Text-Driven Consistent Scene Generation. In NeurIPS; Curran Associates, Inc., 2023; vol 36, pp. 39897–39914. [Google Scholar]
  72. Fridovich-Keil, S.; Yu, A.; Tancik, M.; Chen, Q.; Recht, B.; Kanazawa, A. Plenoxels: Radiance Fields Without Neural Networks; CVPR, 2022. [Google Scholar]
  73. Fridovich-Keil, S.; Meanti, G.; Warburg, F.R.; Recht, B.; Kanazawa, A. K-Planes: Explicit Radiance Fields in Space, Time, and Appearance; CVPR, 2023. [Google Scholar]
  74. Friston, S.; Steed, A. Measuring Latency in Virtual Environments. IEEE Trans. Vis. Comput Graph 2014, 20(4), 616–625. [Google Scholar]
  75. Fu, H.; Cai, B.; Gao, L.; Zhang, L.X.; Wang, J.; Li, C.; Zeng, Q.; Sun, C.; Jia, R.; Zhao, B.; Zhang, H. 3D-FRONT: 3D Furnished Rooms With layOuts and semaNTics. ICCV 2021. [Google Scholar]
  76. Gao, J.; Shen, T.; Wang, Z.; Chen, W.; Yin, K.; Li, D.; Litany, O.; Gojcic, Z.; Fidler, S. GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images. NeurIPS 2022. [Google Scholar] [CrossRef]
  77. Garcin, S.; Walker, T.; Mcdonagh, S.; Pearce, T.; Bilen, H.; He, T.; Wang, K.; Bian, J. Beyond Pixel Histories: World Models with Persistent 3D State. ICML 2026. [Google Scholar] [CrossRef]
  78. Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Nets. NIPS 2014. [Google Scholar] [CrossRef]
  79. Grauman, K.; Westbury, A.; Byrne, E.; Chavis, Z.; Furnari, A.; Girdhar, R.; Hamburger, J.; Jiang, H.; Liu, M.; Liu, X.; Martin, M.; Nagarajan, T.; Radosavovic, I.; Ramakrishnan, S.K.; Ryan, F.; Sharma, J.; Wray, M.; Xu, M.; Xu, E.Z.; Zhao, C.; Bansal, S.; Batra, D.; Cartillier, V.; Crane, S.; Do, T.; Doulaty, M.; Erapalli, A.; Feichtenhofer, C.; Fragomeni, A.; Fu, Q.; Gebreselasie, A.; González, C.; Hillis, J.; Huang, X.; Huang, Y.; Jia, W.; Khoo, W.; Kolář, J.; Kottur, S.; Kumar, A.; Landini, F.; Li, C.; Li, Y.; Li, Z.; Mangalam, K.; Modhugu, R.; Munro, J.; Murrell, T.; Nishiyasu, T.; Price, W.; Ruiz, P.; Ramazanova, M.; Sari, L.; Somasundaram, K.; Southerland, A.; Sugano, Y.; Tao, R.; Vo, M.; Wang, Y.; Wu, X.; Yagi, T.; Zhao, Z.; Zhu, Y.; Arbeláez, P.; Crandall, D.; Damen, D.; Farinella, G.M.; Fuegen, C.; Ghanem, B.; Ithapu, V.K.; Jawahar, C.V.; Joo, H.; Kitani, K.; Li, H.; Newcombe, R.; Oliva, A.; Park, H.S.; Rehg, J.M.; Sato, Y.; Shi, J.; Shou, M.Z.; Torralba, A.; Torresani, L.; Yan, M.; Malik, J. Ego4D: Around the World in 3,000 Hours of Egocentric Video. CVPR 2022, 18995–19012. [Google Scholar]
  80. Grauman, K.; Westbury, A.; Torresani, L.; Kitani, K.; Malik, J.; Afouras, T.; Ashutosh, K.; Baiyya, V.; Bansal, S.; Boote, B.; Byrne, E.; Chavis, Z.; Chen, J.; Cheng, F.; Chu, F.J.; Crane, S.; Dasgupta, A.; Dong, J.; Escobar, M.; Forigua, C.; Gebreselasie, A.; Haresh, S.; Huang, J.; Islam, M.M.; Jain, S.; Khirodkar, R.; Kukreja, D.; Liang, K.J.; Liu, J.W.; Majumder, S.; Mao, Y.; Martin, M.; Mavroudi, E.; Nagarajan, T.; Ragusa, F.; Ramakrishnan, S.K.; Seminara, L.; Somayazulu, A.; Song, Y.; Su, S.; Xue, Z.; Zhang, E.; Zhang, J.; Castillo, A.; Chen, C.; Fu, X.; Furuta, R.; Gonzalez, C.; Gupta, P.; Hu, J.; Huang, Y.; Huang, Y.; Khoo, W.; Kumar, A.; Kuo, R.; Lakhavani, S.; Liu, M.; Luo, M.; Luo, Z.; Meredith, B.; Miller, A.; Oguntola, O.; Pan, X.; Peng, P.; Pramanick, S.; Ramazanova, M.; Ryan, F.; Shan, W.; Somasundaram, K.; Song, C.; Southerland, A.; Tateno, M.; Wang, H.; Wang, Y.; Yagi, T.; Yan, M.; Yang, X.; Yu, Z.; Zha, S.C.; Zhao, C.; Zhao, Z.; Zhu, Z.; Zhuo, J.; Arbelaez, P.; Bertasius, G.; Damen, D.; Engel, J.; Farinella, G.M.; Furnari, A.; Ghanem, B.; Hoffman, J.; Jawahar, C.; Newcombe, R.; Park, H.S.; Rehg, J.M.; Sato, Y.; Savva, M.; Shi, J.; Shou, M.Z.; Wray, M. Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives. CVPR 2024, 19383–19400. [Google Scholar]
  81. Groueix, T.; Fisher, M.; Kim, V.G.; Russell, B.C.; Aubry, M. A Papier-Mâché Approach to Learning 3D Surface Generation; CVPR, 2018. [Google Scholar]
  82. Gu, D.; Lin, J.; Liu, G.; Liu, J.; Ma, S.; Shi, G.; Wang, J.; Wang, Q.; Wu, Q.; Xu, Z.; Yi, X.; Yu, Z.; Zhang, J.; Zheng, Z.; Zhu, Y.; Chen, R.; Guo, H.; Guo, X.; Han, M.; Han, X.; Li, X.; Liang, Y.; Lou, W.; Lu, J.; Luo, G.; Qin, M.; Wang, S.; Wang, Y. Seed3D 2.0: Advancing High-Fidelity Simulation-Ready 3D Content Generation. 2026. [Google Scholar]
  83. Gu, J.; Xiang, F.; Li, X.; Ling, Z.; Liu, X.; Mu, T.; Tang, Y.; Tao, S.; Wei, X.; Yao, Y.; Yuan, X.; Xie, P.; Huang, Z.; Chen, R.; Su, H. ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills. ICLR 2023. [Google Scholar] [CrossRef]
  84. Guo, Y.; Yang, C.; Rao, A.; Liang, Z.; Wang, Y.; Qiao, Y.; Agrawala, M.; Lin, D.; Dai, B. AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning. ICLR 2024, arXiv:2307.04725. [Google Scholar] [CrossRef]
  85. Gupta, K.; Joshi, G.; Chen, Y.R.; Jain, S.; Mehta, I.; Chandraker, M. Reflecting Process Expertise in Procedural Material Generation. In Computer Vision – ECCV 2026, Lecture Notes in Computer Science; Springer, 2026; vol 17046, pp. 318–336. [Google Scholar]
  86. Guédon, A.; Lepetit, V. SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering; CVPR, 2024. [Google Scholar]
  87. Hafner, D.; Lillicrap, T.; Fischer, I.; Villegas, R.; Ha, D.; Lee, H.; Davidson, J. Learning Latent Dynamics for Planning from Pixels. ICML 2019, arXiv:1811.04551. [Google Scholar] [CrossRef]
  88. Hafner, D.; Lillicrap, T.; Ba, J.; Norouzi, M. Dream to Control: Learning Behaviors by Latent Imagination. ICLR 2020, arXiv:1912.01603. [Google Scholar] [CrossRef]
  89. Hafner, D.; Pasukonis, J.; Ba, J.; Lillicrap, T. Mastering diverse control tasks through world models. Nature 2025, 640, 647–653. [Google Scholar]
  90. Han, Y.; Jeong, J. Leveraging generative AI for high-fidelity 360° spatial images: Methodological validation for use as experimental stimuli. Buildings 2026, 16(9), 1679. [Google Scholar]
  91. Haque, A.; Tancik, M.; Efros, A.A.; Holynski, A.; Kanazawa, A. Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions; ICCV, 2023. [Google Scholar]
  92. Hart, J.C. Sphere tracing: a geometric method for the antialiased ray tracing of implicit surfaces. Vis. Comput 1996, 12(10), 527–545. [Google Scholar]
  93. Hart, S.G.; Staveland, L.E. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Advances in Psychology; Hancock, P.A., Meshkati, N., Eds.; 1988; vol 52. North-Holland, pp. 139–183. [Google Scholar]
  94. He, H.; Xu, Y.; Guo, Y.; Wetzstein, G.; DAI, B.; Li, H.; Yang, C. CameraCtrl: Enabling Camera Control for Video Diffusion Models. ICLR 2025a. [Google Scholar] [CrossRef]
  95. He, X.; Peng, C.; Liu, Z.; Wang, B.; Zhang, Y.; Cui, Q.; Kang, F.; Jiang, B.; An, M.; Ren, Y.; Xu, B.; Guo, H.X.; Gong, K.; Wu, C.; Li, W.; Song, X.; Liu, Y.; Li, E.; Zhou, Y. Matrix-game 2.0: An open-source real-time and streaming interactive world model. revised. 2025b. Version v2. [Google Scholar]
  96. Henschel, R.; Khachatryan, L.; Poghosyan, H.; Hayrapetyan, D.; Tadevosyan, V.; Wang, Z.; Navasardyan, S.; Shi, H. StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text; CVPR, 2025; pp. 2568–2577. [Google Scholar]
  97. Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; Hochreiter, S. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. NIPS 2017. [Google Scholar] [CrossRef]
  98. Ho, J.; Salimans, T. Classifier-Free Diffusion Guidance. arXiv 2022, arXiv:2207.12598. [Google Scholar]
  99. Ho, J.; Jain, A.; Abbeel, P. Denoising Diffusion Probabilistic Models. NeurIPS 2020, arXiv:2006.11239. [Google Scholar] [CrossRef]
  100. Ho, J.; Chan, W.; Saharia, C.; Whang, J.; Gao, R.; Gritsenko, A.; Kingma, D.P.; Poole, B.; Norouzi, M.; Fleet, D.J.; Salimans, T. Imagen Video: High Definition Video Generation with Diffusion Models. arXiv 2022a, arXiv:2210.02303. [Google Scholar]
  101. Ho, J.; Salimans, T.; Gritsenko, A.; Chan, W.; Norouzi, M.; Fleet, D.J. Video Diffusion Models; NeurIPS, 2022b; pp. 8633–8646. [Google Scholar]
  102. Hoffman, D.M.; Girshick, A.R.; Akeley, K.; Banks, M.S. Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. J. Vis. 2008, 8(3), 33. [Google Scholar]
  103. Höllein, L.; Cao, A.; Owens, A.; Johnson, J.; Nießner, M. Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models; ICCV, 2023; pp. 7909–7920. [Google Scholar]
  104. Holloway, R.L. Registration Error Analysis for Augmented Reality. Presence (Camb) 1997, 6(4), 413–432. [Google Scholar]
  105. Hong, W.; Ding, M.; Zheng, W.; Liu, X.; Tang, J. CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers. ICLR 2023, arXiv:2205.15868. [Google Scholar] [CrossRef]
  106. Hong, Y.; Zhang, K.; Gu, J.; Bi, S.; Zhou, Y.; Liu, D.; Liu, F.; Sunkavalli, K.; Bui, T.; Tan, H. LRM: Large Reconstruction Model for Single Image to 3D. ICLR 2024. [Google Scholar] [CrossRef]
  107. Hu, B.; Qi, Z.; Huang, G.; Xu, Z.; Zhang, R.; Ye, C.; Zhou, J.; Li, X.; Wang, J. Identity-consistent video generation under large facial-angle variations. 2026. [Google Scholar]
  108. Hu, E.; Li, M.; Hong, J.; Qian, X.; Olwal, A.; Kim, D.; Heo, S.; Du, R. Thing2Reality: Enabling Spontaneous Creation of 3D Objects from 2D Content using Generative AI in XR Meetings; UIST, 2025a. [Google Scholar]
  109. Hu, K.; Jin, Y.; Zhou, H.; Du, L.; Liu, J.; Liu, X. Generative AI for Immersive Video: Recent Advances and Future Opportunities; IJCAI, 2025b; pp. 10464–10472. [Google Scholar]
  110. Hu, Y.; Wang, K.; Shao, Y.; Plass, J.; Wang, Z.; Perlin, K. Generative Terrain Authoring with Mid-air Hand Sketching in Virtual Reality. ACM VRST, 2024. [Google Scholar]
  111. Huang, B.; Yu, Z.; Chen, A.; Geiger, A.; Gao, S. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields. SIGGRAPH Conference Papers, 2024a. [Google Scholar]
  112. Huang, G. Spatio-temporal modeling for action recognition in videos. PhD thesis, University of York, 2022. Available online: https://etheses.whiterose.ac.uk/id/eprint/31262/.
  113. Huang, G.; Bors, A.G. Learning spatio-temporal representations with temporal squeeze pooling. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); IEEE, 2020; pp. 2103–2107. [Google Scholar]
  114. Huang, G.; Bors, A.G. Region-based non-local operation for video classification. In 2020 25th International Conference on Pattern Recognition (ICPR); IEEE, 2021; pp. 10010–10017. [Google Scholar]
  115. Huang, G.; Bors, A.G. BQN: Busy-quiet net enabled by motion band-pass module for action recognition. IEEE Trans. Image Process 2022a, 31, 4966–4979. [Google Scholar]
  116. Huang, G.; Bors, A.G. Busy-quiet video disentangling for video classification. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022b; pp. 1341–1350. [Google Scholar]
  117. Huang, G.; Fu, H.; Bors, A.G. Masked image residual learning for scaling deeper vision transformers. In NeurIPS; Curran Associates, Inc., 2023; vol 36, pp. 57570–57582. [Google Scholar]
  118. Huang, G.; Lin, R.; Li, Y.; Bull, D.R.; Anantrasirichai, N. BVI-Mamba: Video enhancement using a visual state-space model for low-light and underwater environments. In Machine Learning from Challenging Data 2025, Proceedings of SPIE; Markopoulos, P., Ouyang, B., Sklivanitis, G., Eds.; SPIE, 2025a; vol 13460, p. 1346009. [Google Scholar]
  119. Huang, G.; Wang, H.; Qi, Z.; Lu, W.; Bull, D.; Anantrasirichai, N. From Restoration to Reconstruction: Rethinking 3D Gaussian Splatting for Underwater Scenes. arXiv 2025b. [Google Scholar]
  120. Huang, G.; Wang, H.; Seymour, B.; Kovacs, E.; Ellerbroc, J.; Blackham, D.; Anantrasirichai, N. Visual enhancement and 3D representation for underwater scenes: a review. Artif. Intell. Rev. 2026a, 59(10), 205. [Google Scholar]
  121. Huang, G.; Yang, Q.; Lin, R.; Qi, Z.; Bull, D.; Anantrasirichai, N. Bayesian neural networks for one-to-many mapping in image enhancement. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026b; pp. 5004–5012. [Google Scholar]
  122. Huang, S.; Wu, J.; Zhou, Q.; Miao, S.; Long, M. Vid2World: Crafting Video Diffusion Models to Interactive World Models. ICLR 2026c, arXiv:2505.14357. [Google Scholar] [CrossRef]
  123. Huang, T.; Zheng, W.; Wang, T.; Liu, Y.; Wang, Z.; Wu, J.; Jiang, J.; Li, H.; Lau, R.W.H.; Zuo, W.; Guo, C. Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation. ACM Trans. Graph 2025c, 44(6), 1–15. [Google Scholar]
  124. Huang, X.; Li, Z.; He, G.; Zhou, M.; Shechtman, E. Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion. NeurIPS 2025d, arXiv:2506.08009. [Google Scholar] [CrossRef]
  125. Huang, Z.; He, Y.; Yu, J.; Zhang, F.; Si, C.; Jiang, Y.; Zhang, Y.; Wu, T.; Jin, Q.; Chanpaisit, N.; Wang, Y.; Chen, X.; Wang, L.; Lin, D.; Qiao, Y.; Liu, Z. VBench: Comprehensive Benchmark Suite for Video Generative Models. CVPR 2024b. [Google Scholar]
  126. Huang, Z.H.; Lin, G.; Lin, J.; Huang, Y.C.; Yu, R.; Niu, M.; Yang, S.; Liu, Y.L.; Chuang, Y.Y.; Zhang, K.; Wang, Z. Programmable World Model. arXiv 2026d. [Google Scholar]
  127. HunyuanWorld Team; Wang, Z.; Liu, Y.; Wu, J.; Gu, Z.; Wang, H.; Zuo, X.; Huang, T.; Li, W.; Zhang, S.; Lian, Y.; Tsai, Y.; Wang, L.; Liu, S.; Jiang, P.; Yang, X.; Guo, D.; Tang, Y.; Mao, X.; Yu, J.; Yu, J.; Zhang, J.; Chen, M.; Dong, L.; Jia, Y.; Zhang, C.; Tan, Y.; Zhang, H.; Ye, Z.; He, P.; Wu, R.; Chen, M.; Li, Z.; Qin, W.; Wang, L.; Sun, Y.; Niu, L.; Yuan, X.; Yang, X.; He, Y.; Xiao, J.; Tao, Y.; Zhu, J.; Xue, J.; Liu, K.; Zhao, C.; Wu, X.; Liu, T.; Chen, P.; Wang, D.; Liu, Y.; Linus; Jiang, J.; Wang, T.; Guo, C. HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels. technical report. 2025.
  128. Ihara, K.; Lee, D.; Abe, M.; Jo, H.Y.; Suzuki, R. CinemaWorld: Generative Augmented Reality with LLMs and 3D Scene Generation for Movie Augmentation. preprint Version v1;accepted to UIST 2026. 2026. [Google Scholar]
  129. Jain, A.; Mildenhall, B.; Barron, J.T.; Abbeel, P.; Poole, B. Zero-Shot Text-Guided Object Generation With Dream Fields. CVPR 2022. [Google Scholar]
  130. Jerald, J.; Whitton, M.C. Relating Scene-Motion Thresholds to Latency Thresholds for Head-Mounted Displays; IEEE, 2009; Volume VR, pp. 211–218. [Google Scholar]
  131. Jiang, J.; Li, B. PartInteractor: Intent-Driven Part-Aware 3D Authoring for Continuous Co-Creation in XR. preprint Version v1;accepted to UIST 2026. 2026. [Google Scholar]
  132. Jiang, W.; He, W.; Tan, Z.; Kuang, Z.; Yu, D.; Hasegawa, S.; Mayer, S.; Sarsenbayeva, Z. CoGen3D: An Agentic Human-AI Co-Design Pipeline for 3D Asset Generation for Virtual Reality. 2026, Version v1. [Google Scholar]
  133. Jiang, Y.; Zhang, L.; Gao, J.; Hu, W.; Yao, Y. Consistent4D: Consistent 360∘ Dynamic Object Generation from Monocular Video. ICLR 2024. [Google Scholar] [CrossRef]
  134. Jiang, Z.; Wang, H.; Huang, G.; Seymour, B.; Anantrasirichai, N. RUSplatting: Robust 3D Gaussian Splatting for Sparse-View Underwater Scene Reconstruction. In Proceedings of the British Machine Vision Conference, 2025a. [Google Scholar]
  135. Jiang, Z.; Wang, H.; Huang, G.; Seymour, B.; Anantrasirichai, N. Semantic-guided Gaussian Splatting for High-Fidelity Underwater Scene Reconstruction. arXiv, technical version inspected: Version v3;initial preprint 2025; 2025b. [Google Scholar]
  136. Jun, H.; Nichol, A. Shap-E: Generating Conditional 3D Implicit Functions. ArXiv 2023, arXiv:2305.02463. [Google Scholar]
  137. Kajiya, J.T. The rendering equation. SIGGRAPH Comput Graph 1986, 20(4), 143–150. [Google Scholar]
  138. Karras, T.; Laine, S.; Aila, T. A Style-Based Generator Architecture for Generative Adversarial Networks; CVPR, 2019. [Google Scholar]
  139. Kato, H.; Ushiku, Y.; Harada, T. Neural 3D Mesh Renderer. CVPR 2018. [Google Scholar]
  140. Kazhdan, M.; Bolitho, M.; Hoppe, H. Poisson Surface Reconstruction; SGP, 2006; pp. 61–70. [Google Scholar]
  141. Kennedy, R.S.; Lane, N.E.; Berbaum, K.S.; Lilienthal, M.G. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. Int. J. Aviat. Psychol. 1993, 3(3), 203–220. [Google Scholar]
  142. Kerbl, B.; Kopanas, G.; Leimkühler, T.; Drettakis, G. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. Graph 2023, 42(4). [Google Scholar]
  143. Kilteni, K.; Groten, R.; Slater, M. The Sense of Embodiment in Virtual Reality. Presence (Camb) 2012, 21(4), 373–387. [Google Scholar]
  144. Kim, G.; Park, J.; Ryu, N.; Liu, D.; Cho, S. SceneFrom3D: Geometry-Conditioned Outdoor 3D Scene Generation via View Scheduling with Object-Level Control. 2026. [Google Scholar]
  145. Kim, J.; Kang, J.; Choi, J.; Han, B. FIFO-Diffusion: Generating Infinite Videos from Text without Training. NeurIPS 2024, arXiv:2405.11473. [Google Scholar] [CrossRef]
  146. Kingma, D.P.; Welling, M. Auto-Encoding Variational Bayes. ICLR 2014. [Google Scholar] [CrossRef]
  147. Klein, G.; Murray, D. Parallel Tracking and Mapping for Small AR Workspaces. In IEEE ISMAR; 2007; pp. 1–10. [Google Scholar]
  148. Kolve, E.; Mottaghi, R.; Gordon, D.; Zhu, Y.; Gupta, A.; Farhadi, A. AI2-THOR: An Interactive 3D Environment for Visual AI. arXiv 2017, arXiv:1712.05474v1. [Google Scholar]
  149. Kondratyuk, D.; Yu, L.; Gu, X.; Lezama, J.; Huang, J.; Schindler, G.; Hornung, R.; Birodkar, V.; Yan, J.; Chiu, M.C.; Somandepalli, K.; Akbari, H.; Alon, Y.; Cheng, Y.; Dillon, J.; Gupta, A.; Hahn, M.; Hauth, A.; Hendon, D.; Martinez, A.; Minnen, D.; Sirotenko, M.; Sohn, K.; Yang, X.; Adam, H.; Yang, M.H.; Essa, I.; Wang, H.; Ross, D.A.; Seybold, B.; Jiang, L. VideoPoet: A Large Language Model for Zero-Shot Video Generation. ICML 2024, arXiv:2312.14125. [Google Scholar] [CrossRef]
  150. Kong, L.; Yang, Y.; Mei, J.; Liu, Y.; Liang, A.; Zhu, D.; Lu, D.; Yin, W.; Hu, X.; Jia, M.; Deng, J.; Zhang, K.; Wu, Y.; Yan, T.; Gao, S.; Wang, S.; Li, L.; Pan, L.; Liu, Y.; Zhu, J.; Ooi, W.T.; Hoi, S.C.H.; Liu, Z. 3D and 4D World Modeling: A Survey. 2025.
  151. Kong, W.; Tian, Q.; Zhang, Z.; Min, R.; Dai, Z.; Zhou, J.; Xiong, J.; Li, X.; Wu, B.; Zhang, J.; Wu, K.; Lin, Q.; Yuan, J.; Long, Y.; Wang, A.; Wang, A.; Li, C.; Huang, D.; Yang, F.; Tan, H.; Wang, H.; Song, J.; Bai, J.; Wu, J.; Xue, J.; Wang, J.; Wang, K.; Liu, M.; Li, P.; Li, S.; Wang, W.; Yu, W.; Deng, X.; Li, Y.; Chen, Y.; Cui, Y.; Peng, Y.; Yu, Z.; He, Z.; Xu, Z.; Zhou, Z.; Xu, Z.; Tao, Y.; Lu, Q.; Liu, S.; Zhou, D.; Wang, H.; Yang, Y.; Wang, D.; Liu, Y.; Jiang, J.; Zhong, C. HunyuanVideo: A Systematic Framework For Large Video Generative Models. arXiv 2024, arXiv:2412.03603. [Google Scholar]
  152. Kramida, G. Resolving the Vergence-Accommodation Conflict in Head-Mounted Displays. IEEE Trans. Vis. Comput Graph 2016, 22(7), 1912–1931. [Google Scholar]
  153. Lan, Y.; Hong, F.; Yang, S.; Zhou, S.; Meng, X.; Dai, B.; Pan, X.; Loy, C.C. LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation. ECCV Inspected. 2024, arXiv:2403.12019v2. [Google Scholar]
  154. Lee, D.; Sun, Q.; Yoon, S.H. HOICraft: In-situ VLM-based authoring tool for part-level hand-object interaction design in VR. In CHI; Association for Computing Machinery, 2026; pp. 1–25. [Google Scholar]
  155. Lee, J.; Aleotti, F.; Mazala, D.; Garcia-Hernando, G.; Vicente, S.; Johnston, O.J.; Kraus-Liang, I.; Powierza, J.; Shin, D.; Froehlich, J.E.; Brostow, G.; Van Brummelen, J. ImaginateAR: AI-assisted in-situ authoring in augmented reality. In UIST; Association for Computing Machinery, 2025; Volume UIST ’25. [Google Scholar]
  156. Lewis, J.R. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. Int. J. Hum. Comput Interact. 1995, 7(1), 57–78. [Google Scholar]
  157. Li, B.; Yuan, L.P.; Wang, Z. VideoCraft: A Mixed Reality-empowered Video Generation Workflow with Spatial Layer Editing for Concept Video Creation. UIST 2025a. [Google Scholar]
  158. Li, C.; Xia, F.; Martín-Martín, R.; Lingelbach, M.; Srivastava, S.; Shen, B.; Vainio, K.E.; Gokmen, C.; Dharan, G.; Jain, T.; Kurenkov, A.; Liu, K.; Gweon, H.; Wu, J.; Fei-Fei, L.; Savarese, S. iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks; CoRL, 2022. [Google Scholar]
  159. Li, C.; Zhang, R.; Wong, J.; Gokmen, C.; Srivastava, S.; Martín-Martín, R.; Wang, C.; Levine, G.; Lingelbach, M.; Sun, J.; Anvari, M.; Hwang, M.; Sharma, M.; Aydin, A.; Bansal, D.; Hunter, S.; Kim, K.Y.; Lou, A.; Matthews, C.R.; Villa-Renteria, I.; Tang, J.H.; Tang, C.; Xia, F.; Savarese, S.; Gweon, H.; Liu, K.; Wu, J.; Fei-Fei, L. BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation. CoRL 2023. [Google Scholar] [CrossRef]
  160. Li, J.; Tang, J.; Xu, Z.; Wu, L.; Zhou, Y.; Shao, S.; Yu, T.; Cao, Z.; Lu, Q. Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition. ArXiv 2025b, arXiv:2506.17201. [Google Scholar]
  161. Li, L.; Wang, G.; Li, X.; Zhang, Z.; Dou, Q.; Gu, J.; Xue, T.; Shan, Y. CubeComposer: Spatio-Temporal Autoregressive 4K 360∘ Video Generation from Perspective Video. CVPR 2026a. [Google Scholar] [CrossRef]
  162. Li, M.; Patil, A.G.; Xu, K.; Chaudhuri, S.; Khan, O.; Shamir, A.; Tu, C.; Chen, B.; Cohen-Or, D.; Zhang, H. GRAINS: Generative Recursive Autoencoders for INdoor Scenes. ACM Trans. Graph 2019, 38(2). [Google Scholar]
  163. Li, R.; Torr, P.; Vedaldi, A.; Jakab, T. VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory. ICCV 2025c. [Google Scholar]
  164. Li, T.; Tian, Y.; Li, H.; Deng, M.; He, K. Autoregressive Image Generation without Vector Quantization. NeurIPS 2024, arXiv:2406.11838. [Google Scholar] [CrossRef]
  165. Li, X.; He, X.; Zhang, L.; Wu, M.; Li, X.; Liu, Y. A Comprehensive Survey on World Models for Embodied AI. Version 3, revised June 2026. arXiv 2025d, arXiv:2510.16732v3. [Google Scholar]
  166. Li, Y.; Zou, Z.X.; Liu, Z.; Wang, D.; Liang, Y.; Yu, Z.; Liu, X.; Guo, Y.C.; Liang, D.; Ouyang, W.; Cao, Y.P. TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models. ArXiv 2025e, arXiv:2502.06608. [Google Scholar]
  167. Li, Z.; Meng, Z.; Shi, S.; Zhai, M.; Tan, J.; Li, C.; Zhang, K. From Pixels to States: Rethinking Interactive World Models as Game Engines. ArXiv 2026b, arXiv:2607.14076. [Google Scholar]
  168. Lillemark, H.; Huang, B.; Zhan, F.; Du, Y.; Keller, T.A. Flow Equivariant World Models: Structured Memory for Dynamic Environments; ICML, 2026; pp. 73968–74002. [Google Scholar]
  169. Limbert, G.; Karnik, A. ImoGenXR: Investigating the Impact on Creativity Support in a Generative-AI Assisted Immersive Authoring Workflow. ICVR 2025. [Google Scholar]
  170. Lin, C.; Mu, Y. InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior. ICLR 2024. [Google Scholar] [CrossRef]
  171. Lin, C.H.; Gao, J.; Tang, L.; Takikawa, T.; Zeng, X.; Huang, X.; Kreis, K.; Fidler, S.; Liu, M.Y.; Lin, T.Y. Magic3D: High-Resolution Text-to-3D Content Creation; CVPR, 2023; pp. 300–309. [Google Scholar]
  172. Lin, R.; Huang, G.; Anantrasirichai, N. Dynamic weight-based temporal aggregation for low-light video enhancement under extreme noise. In 2026 IEEE International Conference on Image Processing (ICIP); IEEE, 2026a. [Google Scholar]
  173. Lin, R.; Huang, G.; Bull, D.; Anantrasirichai, N. PixIE: Prompted pixel-space low-light image enhancement. 2026b. [Google Scholar]
  174. Lin, R.; Huang, G.; Lin, J.; Sun, Q.; Malyugina, A.; Bull, D.R.; Anantrasirichai, N. (2026c) BVI-RLV: A fully registered dataset for low-light video enhancement. IEEE Transactions on Multimedia Accepted/In press, 3 October 2026. [Google Scholar]
  175. Lin, Y.; Lin, C.; Pan, P.; Yan, H.; Feng, Y.; Mu, Y.; Fragkiadaki, K. PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers. In NeurIPS; Curran Associates, Inc., 2025; vol 38, pp. 35387–35415. [Google Scholar]
  176. Ling, H.; Kim, S.W.; Torralba, A.; Fidler, S.; Kreis, K. Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models; CVPR, 2024a. [Google Scholar]
  177. Ling, L.; Sheng, Y.; Tu, Z.; Zhao, W.; Xin, C.; Wan, K.; Yu, L.; Guo, Q.; Yu, Z.; Lu, Y.; Li, X.; Sun, X.; Ashok, R.; Mukherjee, A.; Kang, H.; Kong, X.; Hua, G.; Zhang, T.; Benes, B.; Bera, A. DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision. CVPR 2024b. [Google Scholar]
  178. Ling, L.; Ge, Y.; Sheng, Y.; Bera, A. 2026a) I-Scene: 3D Instance Models are Implicit Generalizable Spatial Learners; CVPR; pp. 26974–26983.
  179. Ling, L.; Lin, C.H.; Lin, T.Y.; Ding, Y.; Zeng, Y.; Sheng, Y.; Ge, Y.; Liu, M.Y.; Bera, A.; Li, Z. Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generation. ICLR 2026b. [Google Scholar] [CrossRef]
  180. Lipman, Y.; Chen, R.T.Q.; Ben-Hamu, H.; Nickel, M.; Le, M. Flow Matching for Generative Modeling. ICLR 2023, arXiv:2210.02747. [Google Scholar] [CrossRef]
  181. Liu, J.; Li, R.; Zhao, H. Generative AI-assisted visualization prototyping for cultural heritage: A computational framework from 2D planes to 3D immersive scenes. Buildings 2026, 16(16), 3319. [Google Scholar]
  182. Liu, M.; Xu, C.; Jin, H.; Chen, L.; T, M.V.; Xu, Z.; Su, H. One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization. NeurIPS 2023a. [Google Scholar] [CrossRef]
  183. Liu, M.; Shi, R.; Chen, L.; Zhang, Z.; Xu, C.; Wei, X.; Chen, H.; Zeng, C.; Gu, J.; Su, H. One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion. CVPR 2024a. [Google Scholar]
  184. Liu, R.; Wu, R.; Van Hoorick, B.; Tokmakov, P.; Zakharov, S.; Vondrick, C. Zero-1-to-3: Zero-shot One Image to 3D Object. ICCV 2023b. [Google Scholar]
  185. Liu, X.; Gong, C.; Liu, Q. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. ICLR 2023c, arXiv:2209.03003. [Google Scholar] [CrossRef]
  186. Liu, Y.; Lin, C.; Zeng, Z.; Long, X.; Liu, L.; Komura, T.; Wang, W. SyncDreamer: Generating Multiview-consistent Images from a Single-view Image. ICLR 2024b. [Google Scholar] [CrossRef]
  187. Liu, Z.; Feng, Y.; Black, M.J.; Nowrouzezahrai, D.; Paull, L.; Liu, W. MeshDiffusion: Score-based Generative 3D Mesh Modeling. ICLR 2023d. [Google Scholar] [CrossRef]
  188. Long, X.; Guo, Y.C.; Lin, C.; Liu, Y.; Dou, Z.; Liu, L.; Ma, Y.; Zhang, S.H.; Habermann, M.; Theobalt, C.; Wang, W. Wonder3D: Single Image to 3D using Cross-Domain Diffusion. CVPR 2024. [Google Scholar]
  189. Lorensen, W.E.; Cline, H.E. Marching cubes: A high resolution 3D surface construction algorithm. SIGGRAPH Comput Graph 1987, 21(4), 163–169. [Google Scholar]
  190. Lu, C.; Zhou, Y.; Bao, F.; Chen, J.; Li, C.; Zhu, J. DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. NeurIPS 2022, arXiv:2206.00927. [Google Scholar] [CrossRef]
  191. Lu, T.; Yu, M.; Xu, L.; Xiangli, Y.; Wang, L.; Lin, D.; Dai, B. Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering. CVPR 2024. [Google Scholar]
  192. Lu, W.; Chen, H.; Yi, A.; Huang, G.; Chung, Y.Y.; Hu, K.; Wang, Z. (2026a) Learning Fine-Grained Geometry for Sparse-View Splatting via Cascade Depth Loss. IEEE International Conference on Multimedia and Expo arXiv, technical version inspected; 2 April 2026; v2. [Google Scholar]
  193. Lu, Y.; Luo, W.; Tu, P.; Li, H.; Zhu, H.; Yu, Z.; Wang, X.; Chen, X.; Peng, X.; Li, X.; Chen, Z. 4DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Models; CVPR, 2026b; pp. 34322–34332. [Google Scholar]
  194. Ma, B.; Gao, H.; Deng, H.; Luo, Z.; Huang, T.; Tang, L.; Wang, X. You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scale. CVPR 2025. [Google Scholar]
  195. Malyugina, A.; Huang, G.; Ruiz, E.; Leslie, B.; Anantrasirichai, N. Marine snow removal using internally generated pseudo ground truth. In 2025 33rd European Signal Processing Conference (EUSIPCO); IEEE, 2025; pp. 900–904. [Google Scholar]
  196. Mandal, D.; Fu, D.; Miller, J.; Villareal, W.; Peng, X.; Chakravarthula, P. EPIC: Epipolar-consistent 360-degree immersive stereo video generation. 2026, Version v1. [Google Scholar]
  197. Mania, K.; Adelstein, B.D.; Ellis, S.R.; Hill, M.I. Perceptual sensitivity to head tracking latency in virtual environments with varying degrees of scene complexity. In Proceedings of the 1st Symposium on Applied perception in graphics and visualization, 2004; pp. 39–47. [Google Scholar]
  198. McMahan, R.P.; Bowman, D.A.; Zielinski, D.J.; Brady, R.B. Evaluating Display Fidelity and Interaction Fidelity in a Virtual Reality Game. IEEE Trans. Vis. Comput Graph 2012, 18(4), 626–633. [Google Scholar]
  199. Mei, Y.; Hartenstein, S.; Wombacher, J.; Mueller, F.; Gugenheimer, J. ShadAR: Real-time Prompting On-demand Visual Augmentation in Augmented Reality. 2026, preprint Version v1. [Google Scholar]
  200. Merrell, P.; Schkufza, E.; Koltun, V. Computer-generated residential building layouts. ACM Trans. Graph 2010, 29(6), 1–12. [Google Scholar]
  201. Mescheder, L.; Oechsle, M.; Niemeyer, M.; Nowozin, S.; Geiger, A. Occupancy Networks: Learning 3D Reconstruction in Function Space; CVPR, 2019. [Google Scholar]
  202. Michel, O.; Bar-On, R.; Liu, R.; Benaim, S.; Hanocka, R. Text2Mesh: Text-Driven Neural Stylization for Meshes; CVPR, 2022. [Google Scholar]
  203. Mildenhall, B.; Srinivasan, P.P.; Tancik, M.; Barron, J.T.; Ramamoorthi, R.; Ng, R. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV 2020. [Google Scholar]
  204. Milgram, P.; Kishino, F. A Taxonomy of Mixed Reality Visual Displays. IEICE Trans. Inf. Syst. 1994, E77-D(12), 1321–1329. [Google Scholar]
  205. Mine, M.R.; Brooks, FPJr.; Séquin, C.H. Moving Objects in Space: Exploiting Proprioception in Virtual-Environment Interaction; SIGGRAPH, 1997; pp. 19–26. [Google Scholar]
  206. Mirza, M.; Osindero, S. Conditional Generative Adversarial Nets. arXiv 2014, arXiv:1411.1784. [Google Scholar]
  207. Mohammad Khalid, N.; Xie, T.; Belilovsky, E.; Popa, T. CLIP-Mesh: Generating textured meshes from text using pretrained image-text models. SIGGRAPH Asia Conference Papers, 2022; pp. 25:1–25:8. [Google Scholar]
  208. Munkberg, J.; Hasselgren, J.; Shen, T.; Gao, J.; Chen, W.; Evans, A.; Müller, T.; Fidler, S. Extracting Triangular 3D Models, Materials, and Lighting From Images. CVPR 2022. [Google Scholar]
  209. Müller, P.; Wonka, P.; Haegler, S.; Ulmer, A.; Gool, L.V. Procedural modeling of buildings. ACM Trans. Graph 2006, 25(3), 614–623. [Google Scholar]
  210. Müller, T.; Evans, A.; Schied, C.; Keller, A. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph 2022, 41(4), 102:1–102:15. [Google Scholar]
  211. Nash, C.; Ganin, Y.; Eslami, S.M.A.; Battaglia, P. PolyGen: An Autoregressive Generative Model of 3D Meshes; ICML, 2020; pp. 7220–7229. [Google Scholar]
  212. Nguyen, T.; Simoff, S.J.; Qu, Z.; Adam, R.; Catchpoole, D.R.; Kennedy, P.J.; Nguyen, Q.V. Generative artificial intelligence and human artificial intelligence interaction in immersive environments: Research and development directions. Int. J. Hum. Comput Interact. published online. 2026. [Google Scholar]
  213. Ning, X.; Zhuo, Y.; Wang, X.; Sio, C.I.D.; Lee, L.H. When Generative Artificial Intelligence Meets Extended Reality: A Systematic Review. Int. J. Hum. Comput Interact. first published online. 2026, 42(10), 6870–6890. [Google Scholar]
  214. Numan, N.; Rajaram, S.; Kumaravel, B.T.; Marquardt, N.; Wilson, A.D. SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending; UIST, 2024. [Google Scholar]
  215. van den Oord, A.; Kalchbrenner, N.; Kavukcuoglu, K. Pixel Recurrent Neural Networks; ICML, 2016. [Google Scholar]
  216. van den Oord, A.; Vinyals, O.; Kavukcuoglu, K. Neural Discrete Representation Learning. NIPS 2017, arXiv:1711.00937. [Google Scholar] [CrossRef]
  217. Oquab, M.; Darcet, T.; Moutakanni, T.; Vo, H.V.; Szafraniec, M.; Khalidov, V.; Fernandez, P.; Haziza, D.; Massa, F.; El-Nouby, A.; Assran, M.; Ballas, N.; Galuba, W.; Howes, R.; Huang, P.Y.; Li, S.W.; Misra, I.; Rabbat, M.; Sharma, V.; Synnaeve, G.; Xu, H.; Jegou, H.; Mairal, J.; Labatut, P.; Joulin, A.; Bojanowski, P. DINOv2: Learning Robust Visual Features without Supervision. Trans. Mach. Learn Res. 2024. [Google Scholar] [CrossRef]
  218. Padmanaban, N.; Konrad, R.; Stramer, T.; Cooper, E.A.; Wetzstein, G. Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays. Proc. Natl. Acad. Sci. U S A 2017, 114(9), 2183–2188. [Google Scholar]
  219. Parish, Y.I.H.; Müller, P. Procedural modeling of cities; SIGGRAPH, 2001; pp. 301–308. [Google Scholar]
  220. Park, J.J.; Florence, P.; Straub, J.; Newcombe, R.; Lovegrove, S. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation; CVPR, 2019. [Google Scholar]
  221. Parker-Holder, J.; Fruchter, S. 2025) Genie 3: A new frontier for world models. Google DeepMind research blog. 5 August 2025. Available online: https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/ (accessed on 21 October 2026).
  222. Paschalidou, D.; Kar, A.; Shugrina, M.; Kreis, K.; Geiger, A.; Fidler, S. ATISS: Autoregressive Transformers for Indoor Scene Synthesis. In NeurIPS; Curran Associates, Inc., 2021; vol 34, pp. 12013–12026. [Google Scholar]
  223. Peebles, W.; Xie, S. Scalable Diffusion Models with Transformers; ICCV, 2023; pp. 4195–4205. [Google Scholar]
  224. Peng, S.; Niemeyer, M.; Mescheder, L.; Pollefeys, M.; Geiger, A. Convolutional Occupancy Networks. ECCV 2020. [Google Scholar]
  225. Piumsomboon, T.; Lee, G.A.; Hart, J.D.; Ens, B.; Lindeman, R.W.; Thomas, B.H.; Billinghurst, M. Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration; CHI, 2018; pp. 1–13. [Google Scholar]
  226. Po, R.; Nitzan, Y.; Zhang, R.; Chen, B.; Dao, T.; Shechtman, E.; Wetzstein, G.; Huang, X. Long-Context State-Space Video World Models. ICCV 2025. [Google Scholar]
  227. Po, R.; Zhang, D.J.; Hertz, A.; Wetzstein, G.; Wadhwa, N.; Ruiz, N. MultiGen: Level-design for editable multiplayer worlds in diffusion game engines. 2026, Version v2. [Google Scholar]
  228. Polyak, A.; Zohar, A.; Brown, A.; Tjandra, A.; Sinha, A.; Lee, A.; Vyas, A.; Shi, B.; Ma, C.Y.; Chuang, C.Y.; Yan, D.; Choudhary, D.; Wang, D.; Sethi, G.; Pang, G.; Ma, H.; Misra, I.; Hou, J.; Wang, J.; Jagadeesh, K.; Li, K.; Zhang, L.; Singh, M.; Williamson, M.; Le, M.; Yu, M.; Singh, M.K.; Zhang, P.; Vajda, P.; Duval, Q.; Girdhar, R.; Sumbaly, R.; Rambhatla, S.S.; Tsai, S.; Azadi, S.; Datta, S.; Chen, S.; Bell, S.; Ramaswamy, S.; Sheynin, S.; Bhattacharya, S.; Motwani, S.; Xu, T.; Li, T.; Hou, T.; Hsu, W.N.; Yin, X.; Dai, X.; Taigman, Y.; Luo, Y.; Liu, Y.C.; Wu, Y.C.; Zhao, Y.; Kirstain, Y.; He, Z.; He, Z.; Pumarola, A.; Thabet, A.; Sanakoyeu, A.; Mallya, A.; Guo, B.; Araya, B.; Kerr, B.; Wood, C.; Liu, C.; Peng, C.; Vengertsev, D.; Schonfeld, E.; Blanchard, E.; Juefei-Xu, F.; Nord, F.; Liang, J.; Hoffman, J.; Kohler, J.; Fire, K.; Sivakumar, K.; Chen, L.; Yu, L.; Gao, L.; Georgopoulos, M.; Moritz, R.; Sampson, S.K.; Li, S.; Parmeggiani, S.; Fine, S.; Fowler, T.; Petrovic, V.; Du, Y. Movie Gen: A Cast of Media Foundation Models. arXiv 2024, arXiv:2410.13720. [Google Scholar]
  229. Poole, B.; Jain, A.; Barron, J.T.; Mildenhall, B. DreamFusion: Text-to-3D using 2D Diffusion. ICLR 2023. [Google Scholar] [CrossRef]
  230. Poupyrev, I.; Billinghurst, M.; Weghorst, S.; Ichikawa, T. The go-go interaction technique; UIST, 1996; pp. 79–80. [Google Scholar]
  231. Pumarola, A.; Corona, E.; Pons-Moll, G.; Moreno-Noguer, F. D-NeRF: Neural Radiance Fields for Dynamic Scenes. CVPR 2021. [Google Scholar]
  232. Qi, Z.; Huang, G.; Liu, C.; Ye, F. Layered rendering diffusion model for controllable zero-shot image synthesis. In ECCV, Lecture Notes in Computer Science; first online; Springer, 2024; vol 15124, pp. 426–443. [Google Scholar]
  233. Qian, G.; Mai, J.; Hamdi, A.; Ren, J.; Siarohin, A.; Li, B.; Lee, H.Y.; Skorokhodov, I.; Wonka, P.; Tulyakov, S.; Ghanem, B. Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors. ICLR 2024. [Google Scholar] [CrossRef]
  234. Qiu, H.; Xia, M.; Zhang, Y.; He, Y.; Wang, X.; Shan, Y.; Liu, Z. FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling. ICLR 2024, arXiv:2310.15169. [Google Scholar] [CrossRef]
  235. Qiu, Z.; Wu, A. Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Image. CVPR 2026. [Google Scholar] [CrossRef]
  236. Radford, A.; Kim, J.W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; Sutskever, I. Learning Transferable Visual Models From Natural Language Supervision; ICML, 2021; pp. 8748–8763. [Google Scholar]
  237. Rahimi, F.; Sadeghi-Niaraki, A.; Choi, S.M. Generative AI Meets Virtual Reality: A Comprehensive Survey on Applications, Challenges, and Future Direction. IEEE Access 2025, 13, 94893–94909. [Google Scholar]
  238. Raistrick, A.; Lipson, L.; Ma, Z.; Mei, L.; Wang, M.; Zuo, Y.; Kayan, K.; Wen, H.; Han, B.; Wang, Y.; Newell, A.; Law, H.; Goyal, A.; Yang, K.; Deng, J. Infinite Photorealistic Worlds using Procedural Generation. CVPR 2023. [Google Scholar]
  239. Raistrick, A.; Mei, L.; Kayan, K.; Yan, D.; Zuo, Y.; Han, B.; Wen, H.; Parakh, M.; Alexandropoulos, S.; Lipson, L.; Ma, Z.; Deng, J. Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation. CVPR 2024. [Google Scholar]
  240. Ramakrishnan, S.K.; Gokaslan, A.; Wijmans, E.; Maksymets, O.; Clegg, A.; Turner, J.; Undersander, E.; Galuba, W.; Westbury, A.; Chang, A.; Savva, M.; Zhao, Y.; Batra, D. Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI. NeurIPS Datasets Benchmarks Track 2021. [Google Scholar] [CrossRef]
  241. Rasch, J.; Töws, J.; Hirzle, T.; Müller, F.; Schmitz, M. CreepyCoCreator? Investigating AI Representation Modes for 3D Object Co-Creation in Virtual Reality; CHI, 2025. [Google Scholar]
  242. Reizenstein, J.; Shapovalov, R.; Henzler, P.; Sbordone, L.; Labatut, P.; Novotny, D. Common Objects in 3D: Large-Scale Learning and Evaluation of Real-Life 3D Category Reconstruction; ICCV, 2021; pp. 10901–10911. [Google Scholar]
  243. Ren, J.; Pan, L.; Tang, J.; Zhang, C.; Cao, A.; Zeng, G.; Liu, Z. DreamGaussian4D: Generative 4D Gaussian Splatting. 2023, Version v3. [Google Scholar]
  244. Ren, X.; Shen, T.; Huang, J.; Ling, H.; Lu, Y.; Nimier-David, M.; Müller, T.; Keller, A.; Fidler, S.; Gao, J. GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control; CVPR, 2025; pp. 6121–6132. [Google Scholar]
  245. Rezende, D.J.; Mohamed, S.; Wierstra, D. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. ICML 2014, arXiv:1401.4082. [Google Scholar] [CrossRef]
  246. Richardson, E.; Metzer, G.; Alaluf, Y.; Giryes, R.; Cohen-Or, D. TEXTure: Text-Guided Texturing of 3D Shapes. SIGGRAPH Conference Papers, 2023. [Google Scholar]
  247. Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; Ommer, B. High-Resolution Image Synthesis with Latent Diffusion Models. CVPR 2022. [Google Scholar]
  248. Saharia, C.; Chan, W.; Saxena, S.; Li, L.; Whang, J.; Denton, E.L.; Ghasemipour, K.; Gontijo Lopes, R.; Karagol Ayan, B.; Salimans, T.; Ho, J.; Fleet, D.; Norouzi, M. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding; NeurIPS, 2022; pp. 36479–36494. [Google Scholar]
  249. Salimans, T.; Ho, J. Progressive Distillation for Fast Sampling of Diffusion Models. ICLR 2022, arXiv:2202.00512. [Google Scholar] [CrossRef]
  250. Sanchez-Vives, M.V.; Slater, M. From presence to consciousness through virtual reality. Nat. Rev. Neurosci. 2005, 6(4), 332–339. [Google Scholar]
  251. Sand.ai, Teng H, Jia H, Sun L, Li L, Li M, Tang M, Han S, Zhang T, Zhang WQ, Luo W, Kang X, Sun Y, Cao Y, Huang Y, Lin Y, Fang Y, Tao Z, Zhang Z, Wang Z, Liu Z, Shi D, Su G, Sun H, Pan H, Wang J, Sheng J, Cui M, Hu M, Yan M, Yin S, Zhang S, Liu T, Yin X, Yang X, Song X, Hu X, Zhang Y, Li Y MAGI-1: Autoregressive Video Generation at Scale. arXiv, 2025; arXiv:2505.13211.
  252. Saredakis, D.; Szpak, A.; Birckhead, B.; Keage, H.A.D.; Rizzo, A.; Loetscher, T. Factors Associated With Virtual Reality Sickness in Head-Mounted Displays: A Systematic Review and Meta-Analysis. Front Hum. Neurosci. 2020, 14. [Google Scholar]
  253. Savva, G.; Michel, O.; Lu, D.; Waiwitlikhit, S.; Meehan, T.; Mishra, D.; Poddar, S.; Lu, J.; Xie, S. Solaris: Building a Multiplayer Video World Model in Minecraft. arXiv 2026. [Google Scholar]
  254. Savva, M.; Kadian, A.; Maksymets, O.; Zhao, Y.; Wijmans, E.; Jain, B.; Straub, J.; Liu, J.; Koltun, V.; Malik, J.; Parikh, D.; Batra, D. Habitat: A Platform for Embodied AI Research; ICCV, 2019; pp. 9339–9347. [Google Scholar]
  255. Schrittwieser, J.; Antonoglou, I.; Hubert, T.; Simonyan, K.; Sifre, L.; Schmitt, S.; Guez, A.; Lockhart, E.; Hassabis, D.; Graepel, T.; Lillicrap, T.; Silver, D. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature 2020, 588, 604–609. [Google Scholar]
  256. Schubert, T.; Friedmann, F.; Regenbrecht, H. The Experience of Presence: Factor Analytic Insights. Presence (Camb) 2001, 10(3), 266–281. [Google Scholar]
  257. Schwind, V.; Knierim, P.; Haas, N.; Henze, N. Using Presence Questionnaires in Virtual Reality; CHI, 2019; pp. 1–12. [Google Scholar]
  258. Shang, Y.; Tang, Y.; Zhang, X.; Wang, S.; Yan, Y.; Zhang, H.; Zheng, Z.; Zhao, J.; Feng, J.; Gao, C.; Chen, J.; Li, Y. A Survey of Embodied World Models. 2025. Available online: https://fi.ee.tsinghua.edu.cn/public/publications/0940dda4-af15-11f0-9d60-0242ac120002.pdf.
  259. Shen, B.; Xia, F.; Li, C.; Martin-Martin, R.; Fan, L.; Wang, G.; Perez-D’Arpino, C.; Buch, S.; Srivastava, S.; Tchapmi, L.; Tchapmi, M.; Vainio, K.; Wong, J.; Fei-Fei, L.; Savarese, S. iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes. In IEEE/RSJ IROS; 2021a; pp. 7520–7527. [Google Scholar]
  260. Shen, T.; Gao, J.; Yin, K.; Liu, M.Y.; Fidler, S. Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis. NeurIPS 2021b. [Google Scholar] [CrossRef]
  261. Shen, T.; Munkberg, J.; Hasselgren, J.; Yin, K.; Wang, Z.; Chen, W.; Gojcic, Z.; Fidler, S.; Sharp, N.; Gao, J. Flexible Isosurface Extraction for Gradient-Based Mesh Optimization. ACM Trans. Graph 2023, 42(4), 37:1–37:16. [Google Scholar]
  262. Shen, Y.; Li, B.; Huang, J.; Yip, D.; Wang, Z. GaussianShopVR: Facilitating Immersive 3D Authoring Using Gaussian Splatting in VR. UIST 2025. [Google Scholar]
  263. Shi, R.; Chen, H.; Zhang, Z.; Liu, M.; Xu, C.; Wei, X.; Chen, L.; Zeng, C.; Su, H. Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model. arXiv 2023, arXiv:2310.15110. [Google Scholar]
  264. Shi, Y.; Wang, P.; Ye, J.; Mai, L.; Li, K.; Yang, X. MVDream: Multi-view Diffusion for 3D Generation. ICLR 2024. [Google Scholar] [CrossRef]
  265. Shibata, T.; Kim, J.; Hoffman, D.M.; Banks, M.S. The zone of comfort: Predicting visual discomfort with stereo displays. J. Vis. 2011, 11(8), 11–11. [Google Scholar]
  266. Siddiqui, Y.; Alliegro, A.; Artemov, A.; Tommasi, T.; Sirigatti, D.; Rosov, V.; Dai, A.; Nießner, M. MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers. CVPR 2024. [Google Scholar]
  267. Singer, U.; Polyak, A.; Hayes, T.; Yin, X.; An, J.; Zhang, S.; Hu, Q.; Yang, H.; Ashual, O.; Gafni, O.; Parikh, D.; Gupta, S.; Taigman, Y. Make-A-Video: Text-to-Video Generation without Text-Video Data. ICLR 2023a, arXiv:2209.14792. [Google Scholar] [CrossRef]
  268. Singer, U.; Sheynin, S.; Polyak, A.; Ashual, O.; Makarov, I.; Kokkinos, F.; Goyal, N.; Vedaldi, A.; Parikh, D.; Johnson, J.; Taigman, Y. Text-To-4D Dynamic Scene Generation. ICML 2023b. [Google Scholar] [CrossRef]
  269. Sitzmann, V.; Martel, J.N.P.; Bergman, A.W.; Lindell, D.B.; Wetzstein, G. Implicit Neural Representations with Periodic Activation Functions. NeurIPS 2020. [Google Scholar] [CrossRef]
  270. Slater, M. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R Soc. Lond. B Biol. Sci. 2009, 364(1535), 3549–3557. [Google Scholar]
  271. Slater, M.; Wilbur, S. A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments. Presence (Camb) 1997, 6(6), 603–616. [Google Scholar]
  272. Slater, M.; Spanlang, B.; Sanchez-Vives, M.V.; Blanke, O. First Person Experience of Body Transfer in Virtual Reality. PLoS ONE 2010, 5(5), e10564. [Google Scholar]
  273. Sohl-Dickstein, J.; Weiss, E.A.; Maheswaranathan, N.; Ganguli, S. Deep Unsupervised Learning using Nonequilibrium Thermodynamics; ICML, 2015. [Google Scholar]
  274. Song, J.; Meng, C.; Ermon, S. Denoising Diffusion Implicit Models; ICLR, 2021a. [Google Scholar]
  275. Song, K.; Chen, B.; Simchowitz, M.; Du, Y.; Tedrake, R.; Sitzmann, V. History-Guided Video Diffusion. ICML 2025, arXiv:2502.06764. [Google Scholar] [CrossRef]
  276. Song, S.; Lichtenberg, S.P.; Xiao, J. SUN RGB-D: A RGB-D Scene Understanding Benchmark Suite; CVPR, 2015; pp. 567–576. [Google Scholar]
  277. Song, Y.; Sohl-Dickstein, J.; Kingma, D.P.; Kumar, A.; Ermon, S.; Poole, B. Score-Based Generative Modeling through Stochastic Differential Equations; ICLR, 2021b. [Google Scholar]
  278. Song, Y.; Dhariwal, P.; Chen, M.; Sutskever, I. Consistency Models. ICML 2023, arXiv:2303.01469. [Google Scholar] [CrossRef]
  279. Speicher, M.; Hall, B.D.; Nebeling, M. What is Mixed Reality? CHI, 2019; pp. 1–15. [Google Scholar]
  280. Srinidhi, S.; Singh, A.; Lu, E.; Rowe, A. GenAssist: Interactive Prompt-Driven XR Program Generation. In 2026 IEEE Conference on Virtual Reality and 3D User Interfaces (VR); IEEE, 2026; pp. 615–625. [Google Scholar]
  281. Srivastava, S.; Li, C.; Lingelbach, M.; Martín-Martín, R.; Xia, F.; Vainio, K.E.; Lian, Z.; Gokmen, C.; Buch, S.; Liu, K.; Savarese, S.; Gweon, H.; Wu, J.; Fei-Fei, L. BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments; CoRL, 2022. [Google Scholar]
  282. Steinicke, F.; Bruder, G.; Jerald, J.; Frenz, H.; Lappe, M. Estimation of Detection Thresholds for Redirected Walking Techniques. IEEE Trans. Vis. Comput Graph 2010, 16(1), 17–27. [Google Scholar]
  283. Straub, J.; Whelan, T.; Ma, L.; Chen, Y.; Wijmans, E.; Green, S.; Engel, J.J.; Mur-Artal, R.; Ren, C.; Verma, S.; Clarkson, A.; Yan, M.; Budge, B.; Yan, Y.; Pan, X.; Yon, J.; Zou, Y.; Leon, K.; Carter, N.; Briales, J.; Gillingham, T.; Mueggler, E.; Pesqueira, L.; Savva, M.; Batra, D.; Strasdat, H.M.; De Nardi, R.; Goesele, M.; Lovegrove, S.; Newcombe, R. The Replica Dataset: A Digital Replica of Indoor Spaces. arXiv 2019, arXiv:1906.05797. [Google Scholar]
  284. Sun, W.; Chen, S.; Liu, F.; Chen, Z.; Duan, Y.; Zhu, J.; Zhang, J.; Wang, Y. DimensionX: Create Any 3D and 4D Scenes from a Single Image with Decoupled Video Diffusion. ICCV 2025, arXiv:2411.04928v1, 13695–13706. [Google Scholar]
  285. Sun, W.; Zhang, H.; Wang, H.; Wu, J.; Wang, Z.; Wang, Z.; Wang, Y.; Zhang, J.; Wang, T.; Guo, C. WorldPlay: Towards long-term geometric consistency for real-time interactive world modeling. ICML Proc. Mach. Learn. Res. 2026a, vol 306. PMLR, 117501–117523. [Google Scholar]
  286. Sun, Y.T.; Huang, Z.; Niu, Y.; Ma, L.; Cao, Y.P.; Ma, Y.; Qi, X. Stereo World Model: Camera-Guided Stereo Video Generation. CVPR 2026b. [Google Scholar] [CrossRef]
  287. Sutherland, I.E. A head-mounted three dimensional display. In Proceedings of the December 9-11, 1968, fall joint computer conference, part I on - AFIPS ’68 (Fall, part I), 1968; pp. 757–764. [Google Scholar]
  288. Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the Inception Architecture for Computer Vision; CVPR, 2016. [Google Scholar]
  289. Szot, A.; Clegg, A.; Undersander, E.; Wijmans, E.; Zhao, Y.; Turner, J.; Maestre, N.; Mukadam, M.; Chaplot, D.S.; Maksymets, O.; Gokaslan, A.; Vondruš, V.; Dharur, S.; Meier, F.; Galuba, W.; Chang, A.; Kira, Z.; Koltun, V.; Malik, J.; Savva, M.; Batra, D. Habitat 2.0: Training Home Assistants to Rearrange their Habitat; NeurIPS, 2021; pp. 251–266. [Google Scholar]
  290. Taha, K. Generative AI for multimodal content: a survey with empirical and experimental evaluations. Artif. Intell. Rev. 2026, 59, article 140. [Google Scholar]
  291. Tamir, N.Y.; Amir, S.; Itzhaky, R.; Atia, N.; Sundaram, S.; Fu, S.; Sokolovsky, R.; Isola, P.; Dekel, T.; Zhang, R.; Farber, M. What Makes for a Good Stereoscopic Image? In CVPR Workshops; 2025; pp. 261–272. [Google Scholar]
  292. Tancik, M.; Srinivasan, P.P.; Mildenhall, B.; Fridovich-Keil, S.; Raghavan, N.; Singhal, U.; Ramamoorthi, R.; Barron, J.T.; Ng, R. Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. NeurIPS 2020. [Google Scholar] [CrossRef]
  293. Tang, J.; Chen, Z.; Chen, X.; Wang, T.; Zeng, G.; Liu, Z. LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation. ECCV 2024a. [Google Scholar]
  294. Tang, J.; Nie, Y.; Markhasin, L.; Dai, A.; Thies, J.; Nießner, M. DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis. CVPR 2024b. [Google Scholar]
  295. Tang, J.; Ren, J.; Zhou, H.; Liu, Z.; Zeng, G. DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation. ICLR 2024c. [Google Scholar] [CrossRef]
  296. Tang, J.; Liu, J.; Li, J.; Wu, L.; Yang, H.; Zhao, P.; Gong, S.; Yuan, X.; Shao, S.; Zhang, L.; Lu, Q. Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model. ArXiv 2025, arXiv:2511.23429. [Google Scholar]
  297. Tang, S.; Zhang, F.; Chen, J.; Wang, P.; Furukawa, Y. MVDiffusion: Enabling holistic multi-view image generation with correspondence-aware diffusion. In NeurIPS; Curran Associates, Inc., 2023; vol 36, pp. 51202–51233. [Google Scholar]
  298. Team HY-World; Cao, C.; Zuo, X.; Wang, Z.; Zhang, Y.; Wu, J.; Liu, Z.; Gong, Y.; Liu, Y.; Yuan, B.; Zhang, C.; Li, C.; Guo, D.; Yang, F.; Zhang, H.; Cao, H.; Zhu, J.; Lin, J.; Xiao, J.; Zhang, J.; Yu, J.; Wang, L.; Wang, L.; Wang, L.; Linus; Chen, M.; He, P.; Zhao, P.; Chen, Q.; Chen, R.; Shao, R.; Liu, S.; Qin, W.; Niu, X.; Yuan, X.; Sun, Y.; Tang, Y.; Sun, Y.; Lian, Y.; Tan, Y.; Liu, Y.; Yin, Y.; Min, Z.; Wang, T.; Guo, C. HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds. 2026, Version v1. [Google Scholar]
  299. Tong, Y.; Qiu, Y.; Li, R.; Qiu, S.; Heng, P.A. MS2Mesh-XR: Multi-Modal Sketch-to-Mesh Generation in XR Environments. In IEEE AIxVR. IEEE; 2025; pp. 272–276. [Google Scholar]
  300. Unterthiner, T.; van Steenkiste, S.; Kurach, K.; Marinier, R.; Michalski, M.; Gelly, S. Towards Accurate Generative Models of Video: A New Metric & Challenges. ArXiv 2018, arXiv:1812.01717. [Google Scholar]
  301. Usoh, M.; Catena, E.; Arman, S.; Slater, M. Using Presence Questionnaires in Reality. Presence (Camb) 2000, 9(5), 497–503. [Google Scholar]
  302. Vachha, C.; Kang, Y.; Dive, Z.; Chidambaram, A.; Gupta, A.; Jun, E.; Hartmann, B. Dreamcrafter: Immersive Editing of 3D Radiance Fields Through Flexible, Generative Inputs and Outputs. CHI 2025. [Google Scholar]
  303. Valevski, D.; Leviathan, Y.; Arar, M.; Fruchter, S. Diffusion Models Are Real-Time Game Engines. ICLR 2025. [Google Scholar] [CrossRef]
  304. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser; Polosukhin, I. Attention Is All You Need. NIPS 2017, arXiv:1706.03762. [Google Scholar] [CrossRef]
  305. Verbin, D.; Hedman, P.; Mildenhall, B.; Zickler, T.; Barron, J.T.; Srinivasan, P.P. Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields. CVPR 2022. [Google Scholar]
  306. Villegas, R.; Babaeizadeh, M.; Kindermans, P.J.; Moraldo, H.; Zhang, H.; Saffar, M.T.; Castro, S.; Kunze, J.; Erhan, D. Phenaki: Variable Length Video Generation from Open Domain Textual Descriptions. ICLR 2023, arXiv:2210.02399. [Google Scholar] [CrossRef]
  307. Wan Team; Wang, A.; Ai, B.; Wen, B.; Mao, C.; Xie, C.W.; Chen, D.; Yu, F.; Zhao, H.; Yang, J.; Zeng, J.; Wang, J.; Zhang, J.; Zhou, J.; Wang, J.; Chen, J.; Zhu, K.; Zhao, K.; Yan, K.; Huang, L.; Feng, M.; Zhang, N.; Li, P.; Wu, P.; Chu, R.; Feng, R.; Zhang, S.; Sun, S.; Fang, T.; Wang, T.; Gui, T.; Weng, T.; Shen, T.; Lin, W.; Wang, W.; Wang, W.; Zhou, W.; Wang, W.; Shen, W.; Yu, W.; Shi, X.; Huang, X.; Xu, X.; Kou, Y.; Lv, Y.; Li, Y.; Liu, Y.; Wang, Y.; Zhang, Y.; Huang, Y.; Li, Y.; Wu, Y.; Liu, Y.; Pan, Y.; Zheng, Y.; Hong, Y.; Shi, Y.; Feng, Y.; Jiang, Z.; Han, Z.; Wu, Z.F.; Liu, Z. Wan: Open and Advanced Large-Scale Video Generative Models. arXiv 2025, arXiv:2503.20314. [Google Scholar]
  308. Wang, D.; Qiu, L.; Cai, L.; Jiang, Y.; Fan, M.; Wang, Z. Visual Generative Artificial Intelligence (GenAI) in Mixed Reality (MR): A Survey of Techniques, Interaction Designs, and Evaluations. Int. J. Hum. Comput Interact. published online. 2026a. [Google Scholar]
  309. Wang, H.; Cai, S.; Azzarelli, A.; Jiang, Z.; Huang, G.; Khoo, E.T.; Seymour, B.; Zhang, F.; Bull, D.; Anantrasirichai, N. (2026b) OceanXL: Large-scale Underwater 3D Gaussian Splatting via Block Partitioning and Adaptive Pruning. SIGGRAPH Asia 2026 Conf. Pap.;arXiv, accepted; technical version inspected; 2026, arXiv:2609.29985v1v1. [Google Scholar]
  310. Wang, H.; Huang, G.; Zhang, F.; Bull, D.; Anantrasirichai, N. Prune Wisely, Reconstruct Sharply: Compact 3D Gaussian Splatting via Adaptive Pruning and Difference-of-Gaussian Primitives; CVPR, 2026c; pp. 11716–11725. [Google Scholar]
  311. Wang, H.; Ouyang, H.; Wang, Q.; Wang, W.; Bai, Q.; Cheng, K.L.; Yu, Y.; Li, Y.; Meng, Y.; Liu, Z.; Zeng, Y.; Shen, Y.; Chen, Q. WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory. arXiv 2026d. [Google Scholar]
  312. Wang, K.; Savva, M.; Chang, A.X.; Ritchie, D. Deep convolutional priors for indoor scene synthesis. ACM Trans. Graph 2018, 37(4), 1–14. [Google Scholar]
  313. Wang, K.; Lin, Y.A.; Weissmann, B.; Savva, M.; Chang, A.X.; Ritchie, D. PlanIT: Planning and Instantiating Indoor Scenes with Relation Graph and Spatial Prior Networks. ACM Trans. Graph 2019, 38(4), 1–15. [Google Scholar]
  314. Wang, L.; Frisvad, J.R.; Jensen, M.B.; Bigdeli, S.A. StereoDiffusion: Training-Free Stereo Image Generation Using Latent Diffusion Models. In CVPR Workshops (GCV); 2024a. [Google Scholar]
  315. Wang, P.; Liu, L.; Liu, Y.; Theobalt, C.; Komura, T.; Wang, W. NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction; NeurIPS, 2021a. [Google Scholar]
  316. Wang, Q.; Wang, Z.; Genova, K.; Srinivasan, P.P.; Zhou, H.; Barron, J.T.; Martin-Brualla, R.; Snavely, N.; Funkhouser, T. IBRNet: Learning Multi-View Image-Based Rendering; CVPR, 2021b. [Google Scholar]
  317. Wang, Q.; Li, W.; Mou, C.; Cheng, X.; Zhang, J. 360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model. CVPR 2024b. [Google Scholar]
  318. Wang, X.; Yuan, H.; Zhang, S.; Chen, D.; Wang, J.; Zhang, Y.; Shen, Y.; Zhao, D.; Zhou, J. VideoComposer: Compositional Video Synthesis with Motion Controllability; NeurIPS, 2023a; pp. 7594–7611. [Google Scholar]
  319. Wang, X.; Cen, Q.; Bi, W.; Ma, Y.; Yi, X.; Xiao, R.; Fu, X.; Li, H. Roomify: Spatially-Grounded Style Transformation for Immersive Virtual Environments; CHI, 2026e. [Google Scholar]
  320. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Trans. Image Process 2004, 13(4), 600–612. [Google Scholar]
  321. Wang, Z.; Lu, C.; Wang, Y.; Bao, F.; Li, C.; Su, H.; Zhu, J. ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation. NeurIPS 2023b. [Google Scholar] [CrossRef]
  322. Wang, Z.; Wang, Y.; Chen, Y.; Xiang, C.; Chen, S.; Yu, D.; Li, C.; Su, H.; Zhu, J. CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model. In ECCV; Springer Nature Switzerland, 2024c; pp. 57–74. [Google Scholar]
  323. Wang, Z.; Yuan, Z.; Wang, X.; Li, Y.; Chen, T.; Xia, M.; Luo, P.; Shan, Y. MotionCtrl: A Unified and Flexible Motion Controller for Video Generation. SIGGRAPH Conference Papers, 2024d. [Google Scholar]
  324. Wang, Z.; Lin, H.; Yoon, J.; Cho, J.; Zhang, Y.; Bansal, M. AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories. In Computer Vision – ECCV; 2026f; Volume 2026, pp. 203–224. [Google Scholar]
  325. Wang, Z.; Liu, Z.; Li, J.; Huang, K.; Xu, B.; Kang, F.; An, M.; Wang, P.; Jiang, B.; Wei, Y.; Xietian, Y.; Pei, J.; Hu, L.; Jiang, B.; Xue, H.; Wang, Z.; Sun, H.; Li, W.; Ouyang, W.; He, X.; Liu, Y.; Li, Y.; Zhou, Y. Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory. posted. 2026g. Version v1. [Google Scholar]
  326. Warburton, M.; Mon-Williams, M.; Mushtaq, F.; Morehead, J.R. Measuring motion-to-photon latency for sensorimotor experiments with virtual reality systems. Behav. Res. Methods 2023, 55(7), 3658–3678. [Google Scholar]
  327. van Waveren, J.M.P. The asynchronous time warp for virtual reality on consumer hardware. In ACM VRST; 2016; pp. 37–46. [Google Scholar]
  328. Weech, S.; Kenny, S.; Barnett-Cowan, M. Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review. Front Psychol. 2019, 10. [Google Scholar]
  329. Welch, G.; Foxlin, E. Motion tracking: no silver bullet, but a respectable arsenal. IEEE Comput Graph Appl. 2002, 22(6), 24–38. [Google Scholar]
  330. Weng, S.C.C.; Chiou, Y.M.; Do, E.Y.L. Dream Mesh: A Speech-to-3D Model Generative Pipeline in Mixed Reality. IEEE AIxVR, 2024. [Google Scholar]
  331. Weng, S.C.C.; Ma, S.Y.; Reinig, S.; Kadam, P.; Zhao, A.Y.; Banić, A.; Suzuki, R.; Do, E.Y.L. Editing Reality: Designing In-Situ Co-Creation with Generative AI in Mixed Reality. DIS 2026. [Google Scholar]
  332. Witmer, B.G.; Singer, M.J. Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence (Camb) 1998, 7(3), 225–240. [Google Scholar]
  333. Wu, J.; Zhang, C.; Xue, T.; Freeman, W.T.; Tenenbaum, J.B. Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling. NIPS 2016. [Google Scholar] [CrossRef]
  334. Wu, J.; Yin, S.; Feng, N.; He, X.; Li, D.; Hao, J.; Long, M. iVideoGPT: Interactive VideoGPTs are Scalable World Models. NeurIPS 2024a, arXiv:2405.15223. [Google Scholar] [CrossRef]
  335. Wu, J.Z.; Ge, Y.; Wang, X.; Lei, S.W.; Gu, Y.; Shi, Y.; Hsu, W.; Shan, Y.; Qie, X.; Shou, M.Z. Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation; ICCV, 2023; pp. 7623–7633. [Google Scholar]
  336. Wu, K.; Liu, F.; Cai, Z.; Yan, R.; Wang, H.; Hu, Y.; Duan, Y.; Ma, K. Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image. NeurIPS 2024b. [Google Scholar]
  337. Wu, R.; Gao, R.; Poole, B.; Trevithick, A.; Zheng, C.; Barron, J.T.; Holynski, A. CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models. CVPR 2025a. [Google Scholar]
  338. Wu, R.; Ye, F.; Liu, Q.; Huang, G.; Guo, J.; Hu, R. Incrementally learning multiple diverse data domains via multi-source dynamic expansion model. 2025b. [Google Scholar]
  339. Wu, T.; Yang, S.; Po, R.; Xu, Y.; Liu, Z.; Lin, D.; Wetzstein, G. Video World Models with Long-term Spatial Memory. NeurIPS 2025c. [Google Scholar] [CrossRef]
  340. Xia, F.; Zamir, A.R.; He, Z.; Sax, A.; Malik, J.; Savarese, S. Gibson Env: Real-World Perception for Embodied Agents; CVPR, 2018; pp. 9068–9079. [Google Scholar]
  341. Xia, H.; Li, X.; Li, Z.; Ma, Q.; Xu, J.; Liu, M.Y.; Cui, Y.; Lin, T.Y.; Ma, W.C.; Wang, S.; Song, S.; Wei, F. SAGE: Scalable Agentic 3D Scene Generation for Embodied AI; CVPR, 2026; pp. 22358–22368. [Google Scholar]
  342. Xiang, F.; Qin, Y.; Mo, K.; Xia, Y.; Zhu, H.; Liu, F.; Liu, M.; Jiang, H.; Yuan, Y.; Wang, H.; Yi, L.; Chang, A.X.; Guibas, L.J.; Su, H. SAPIEN: A SimulAted Part-Based Interactive ENvironment; CVPR, 2020; pp. 11097–11107. [Google Scholar]
  343. Xiang, J.; Liu, G.; Gu, Y.; Gao, Q.; Ning, Y.; Zha, Y.; Feng, Z.; Tao, T.; Hao, S.; Shi, Y.; Liu, Z.; Xing, E.P.; Hu, Z. Pandora: Towards General World Model with Natural Language Actions and Video States. arXiv 2024, arXiv:2406.09455. [Google Scholar]
  344. Xiang, J.; Lv, Z.; Xu, S.; Deng, Y.; Wang, R.; Zhang, B.; Chen, D.; Tong, X.; Yang, J. Structured 3D Latents for Scalable and Versatile 3D Generation; CVPR, 2025; pp. 21469–21480. [Google Scholar]
  345. Xiang, J.; Chen, X.; Xu, S.; Wang, R.; Lv, Z.; Deng, Y.; Zhu, H.; Dong, Y.; Zhao, H.; Yuan, N.J.; Yang, J. Native and Compact Structured Latents for 3D Generation; CVPR, 2026; pp. 14419–14429. [Google Scholar]
  346. Xiao, Z.; Lan, Y.; Zhou, Y.; Ouyang, W.; Yang, S.; Zeng, Y.; Pan, X. WorldMem: Long-term Consistent World Simulation with Memory. NeurIPS Inspected. 2025, arXiv:2504.12369v2. [Google Scholar] [CrossRef]
  347. Xie, H.; Wen, B.; Chen, Z.; Hong, F.; Liu, Z. 3D scene generation: A survey. Int. J. Comput Vis. 2026a, 134, article 418. [Google Scholar]
  348. Xie, L.; Sun, L.C.; Neall, A.; Wu, T.; Cai, S.; Wetzstein, G. Generated Reality: Human-centric world simulation using interactive video generation with hand and camera control. CVPR Find., technical version inspected; 2026b, arXiv:2602.18422v1. [Google Scholar]
  349. Xie, Y.; Yao, C.H.; Voleti, V.; Jiang, H.; Jampani, V. SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency. ICLR 2025. [Google Scholar] [CrossRef]
  350. Xing, J.; Xia, M.; Zhang, Y.; Chen, H.; Yu, W.; Liu, H.; Liu, G.; Wang, X.; Shan, Y.; Wong, T.T. DynamiCrafter: Animating Open-Domain Images with Video Diffusion Priors. ECCV 2024. [Google Scholar]
  351. Xing, K.; Li, L.; Yin, Y.; Liang, H.; Luo, G.; Fang, C.; Wang, J.; Plataniotis, K.N.; Jin, X.; Zhao, Y.; Wei, Y. StereoWorld: Geometry-Aware Monocular-to-Stereo Video Generation. CVPR 2026. [Google Scholar] [CrossRef]
  352. Xiong, Z.; Song, Y.; Kang, H.; Yan, Q.; Jiang, L.; Yang, J.; Fu, Z.; Fotiadis, S.; Wang, A.; Liu, Z.; Liu, B.; Yang, Y.; Lu, X.; Jacobs, N. ActWorld: From Explorable to Interactive World Model via Action-Aware Memory. arXiv 2026. [Google Scholar]
  353. Xiu, Y.; Chilukuri, J.; Sen, S.; Gorlatova, M. Say It, See It: A Systematic Evaluation on Speech-Based 3D Content Generation Methods in Augmented Reality. IEEE ISMAR Adjunct (UNAI workshop), 2025. [Google Scholar]
  354. Xu, J.; Cheng, W.; Gao, Y.; Wang, X.; Gao, S.; Shan, Y. InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models. ArXiv 2024a, arXiv:2404.07191. [Google Scholar]
  355. Xu, J.; Zou, X.; Huang, K.; Chen, Y.; Liu, B.; Cheng, M.; Shi, X.; Huang, J. EasyAnimate: A High-Performance Long Video Generation Method based on Transformer Architecture. arXiv, Version v1; 2024b. [Google Scholar]
  356. Xu, J.; Shao, S.; Bai, L.; Zhou, Z.; Huang, G.; Anantrasirichai, N.; Xie, Z. Golden anchor for video generation. 2026a. [Google Scholar]
  357. Xu, R.; Xue, T.; Dong, Q.; Wan, L.; Zhu, Z.; Li, P.; Dou, Z.; Lin, C.; Xin, S.; Liu, Y.; Wang, W.; Komura, T. MeshMosaic: Scaling Artist Mesh Generation via Local-to-Global Assembly; CVPR, 2026b; pp. 20003–20013. [Google Scholar]
  358. Xu, Y.; Shi, Z.; Wang, Y.; Chen, H.; Yang, C.; Peng, S.; Shen, Y.; Wetzstein, G. GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation. ECCV 2024c. [Google Scholar]
  359. Yan, W.; Zhang, Y.; Abbeel, P.; Srinivas, A. VideoGPT: Video Generation using VQ-VAE and Transformers. arXiv 2021, arXiv:2104.10157. [Google Scholar]
  360. Yang, B.; Dong, W.; Ma, L.; Hu, W.; Liu, X.; Cui, Z.; Ma, Y. DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation. In IEEE VR; 2024a. [Google Scholar]
  361. Yang, G.; Huang, X.; Hao, Z.; Liu, M.Y.; Belongie, S.; Hariharan, B. PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows; ICCV, 2019. [Google Scholar]
  362. Yang, S.; Huang, W.; Chu, R.; Xiao, Y.; Zhao, Y.; Wang, X.; Li, M.; Xie, E.; Chen, Y.; Lu, Y.; Han, S.; Chen, Y. LongLive: Real-time Interactive Long Video Generation; ICLR, 2026; pp. 90037–90059. [Google Scholar]
  363. Yang, X.; Man, Y.; Chen, J.K.; Wang, Y.X. SceneCraft: Layout-Guided 3D Scene Generation; NeurIPS, 2024b. [Google Scholar]
  364. Yang, Y.; Jia, B.; Zhi, P.; Huang, S. PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI. CVPR 2024c. [Google Scholar]
  365. Yang, Y.; Sun, F.Y.; Weihs, L.; VanderBilt, E.; Herrasti, A.; Han, W.; Wu, J.; Haber, N.; Krishna, R.; Liu, L.; Callison-Burch, C.; Yatskar, M.; Kembhavi, A.; Clark, C. Holodeck: Language Guided Generation of 3D Embodied AI Environments; CVPR, 2024d; pp. 16227–16237. [Google Scholar]
  366. Yang, Y.; Guo, Y.C.; Huang, Y.; Zou, Z.X.; Yu, Z.; Li, Y.; Cao, Y.P.; Liu, X. HoloPart: Generative 3D Part Amodal Segmentation. ArXiv 2025a, arXiv:2504.07943. [Google Scholar]
  367. Yang, Z.; Teng, J.; Zheng, W.; Ding, M.; Huang, S.; Xu, J.; Yang, Y.; Hong, W.; Zhang, X.; Feng, G.; Yin, D.; Zhang, Y.; Wang, W.; Cheng, Y.; Xu, B.; Gu, X.; Dong, Y.; Tang, J. CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer. ICLR 2025b. [Google Scholar] [CrossRef]
  368. Yao, Z.; Yao, X.; Sun, Q.; Xiong, H.; Lau, K.W.; Mi, H. Gesture2Prop: Conditional VR Prop Generation from Hand Gestures and Speech. UIST Adjun. 2025. [Google Scholar]
  369. Yariv, L.; Kasten, Y.; Moran, D.; Galun, M.; Atzmon, M.; Basri, R.; Lipman, Y. Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance. NeurIPS 2020. [Google Scholar] [CrossRef]
  370. Yariv, L.; Gu, J.; Kasten, Y.; Lipman, Y. Volume Rendering of Neural Implicit Surfaces; NeurIPS, 2021. [Google Scholar]
  371. Ye, H.; Zhang, J.; Liu, S.; Han, X.; Yang, W. IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models. ArXiv 2023, arXiv:2308.06721. [Google Scholar]
  372. Ye, T.; Mao, Y.; Liao, M.; Liu, J.; Guo, C.; Du, D.; Shou, Q.; Zhu, F.; Guo, S. 3D generation for embodied AI and robotic simulation: A survey. 2026, Version v3. [Google Scholar]
  373. Yeshwanth, C.; Liu, Y.C.; Nießner, M.; Dai, A. ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes; ICCV, 2023; pp. 12–22. [Google Scholar]
  374. Yin, T.; Zhang, Q.; Zhang, R.; Freeman, W.T.; Durand, F.; Shechtman, E.; Huang, X. From Slow Bidirectional to Fast Autoregressive Video Diffusion Models. CVPR 2025. [Google Scholar]
  375. Yu, A.; Ye, V.; Tancik, M.; Kanazawa, A. pixelNeRF: Neural Radiance Fields From One or Few Images. CVPR 2021. [Google Scholar]
  376. Yu, H.X.; Duan, H.; Hur, J.; Sargent, K.; Rubinstein, M.; Freeman, W.T.; Cole, F.; Sun, D.; Snavely, N.; Wu, J.; Herrmann, C. WonderJourney: Going from Anywhere to Everywhere; CVPR, 2024a; pp. 6658–6667. [Google Scholar]
  377. Yu, H.X.; Duan, H.; Herrmann, C.; Freeman, W.T.; Wu, J. WonderWorld: Interactive 3D Scene Generation from a Single Image. CVPR 2025a. [Google Scholar]
  378. Yu, J.; Qin, Y.; Wang, X.; Wan, P.; Zhang, D.; Liu, X. GameFactory: Creating New Games with Generative Interactive Videos. ICCV 2025b. [Google Scholar]
  379. Yu, L.; Cheng, Y.; Sohn, K.; Lezama, J.; Zhang, H.; Chang, H.; Hauptmann, A.G.; Yang, M.H.; Hao, Y.; Essa, I.; Jiang, L. MAGVIT: Masked Generative Video Transformer. CVPR 2023. [Google Scholar]
  380. Yu, W.; Xing, J.; Yuan, L.; Hu, W.; Li, X.; Huang, Z.; Gao, X.; Wong, T.T.; Shan, Y.; Tian, Y. ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 2025c. [Google Scholar]
  381. Yu, Z.; Chen, A.; Huang, B.; Sattler, T.; Geiger, A. Mip-Splatting: Alias-free 3D Gaussian Splatting. CVPR 2024b. [Google Scholar]
  382. Yuan, H.; Chen, W.; Cen, J.; Yu, H.; Liang, J.; Chang, S.; Lin, Z.; Feng, T.; Liu, P.; Xing, J.; Luo, H.; Tang, J.; Wang, F.; Yang, Y. Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspective. ICLR 2026a, arXiv:2507.08801. [Google Scholar] [CrossRef]
  383. Yuan, J.; Yang, B.; Wang, K.; Pan, P.; Ma, L.; Zhang, X.; Liu, X.; Cui, Z.; Ma, Y. ImmerseGen: Agent-Guided Immersive World Generation with Alpha-Textured Proxies. IEEE Trans. Vis. Comput Graph 2026b, 32(5), 3530–3540. [Google Scholar]
  384. Zeng, X.; Vahdat, A.; Williams, F.; Gojcic, Z.; Litany, O.; Fidler, S.; Kreis, K. LION: Latent Point Diffusion Models for 3D Shape Generation. NeurIPS 2022. [Google Scholar] [CrossRef]
  385. Zeng, X.; Chen, X.; Qi, Z.; Liu, W.; Zhao, Z.; Wang, Z.; Fu, B.; Liu, Y.; Yu, G. Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models. CVPR 2024. [Google Scholar]
  386. Zhai, G.; Örnek, E.P.; Chen, D.Z.; Liao, R.; Di, Y.; Navab, N.; Tombari, F.; Busam, B. EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusion. ECCV 2024. [Google Scholar]
  387. Zhang, C.; Wu, Q.; Gambardella, C.C.; Huang, X.; Phung, D.; Ouyang, W.; Cai, J. Taming Stable Diffusion for Text to 360∘ Panorama Image Generation. CVPR 2024a. [Google Scholar]
  388. Zhang, H.; Chen, P.; Xie, X.; Jiang, Z.; Wu, Y.; Li, Z.; Chen, X.; Sun, L. FusionProtor: A Mixed-Prototype Tool for Component-level Physical-to-Virtual 3D Transition and Simulation. CHI 2025. [Google Scholar]
  389. Zhang, J.; Li, X.; Wan, Z.; Wang, C.; Liao, J. Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields. IEEE Trans. Vis. Comput Graph 2024b, 30(12), 7749–7762. [Google Scholar]
  390. Zhang, L.; Rao, A.; Agrawala, M. Adding Conditional Control to Text-to-Image Diffusion Models; ICCV, 2023; pp. 3836–3847. [Google Scholar]
  391. Zhang, L.; Pan, J.; Gettig, J.; Oney, S.; Guo, A. VRCopilot: Authoring 3D Layouts with Generative AI Models in VR. In UIST; Association for Computing Machinery, 2024c; pp. 1–13. [Google Scholar]
  392. Zhang, R.; Isola, P.; Efros, A.A.; Shechtman, E.; Wang, O. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. CVPR 2018. [Google Scholar]
  393. Zhang, X.; Srinivasan, P.P.; Deng, B.; Debevec, P.; Freeman, W.T.; Barron, J.T. NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination. ACM Trans. Graph 2021, 40(6). [Google Scholar]
  394. Zhao, S.; Hu, W.; Cun, X.; Zhang, Y.; Li, X.; Kong, Z.; Gao, X.; Niu, M.; Shan, Y. StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videos. ArXiv 2024, arXiv:2409.07447. [Google Scholar]
  395. Zhao, Z.; Lai, Z.; Lin, Q.; Zhao, Y.; Liu, H.; Yang, S.; Feng, Y.; Yang, M.; Zhang, S.; Yang, X.; Shi, H.; Liu, S.; Wu, J.; Lian, Y.; Yang, F.; Tang, R.; He, Z.; Wang, X.; Liu, J.; Zuo, X.; Chen, Z.; Lei, B.; Weng, H.; Xu, J.; Zhu, Y.; Liu, X.; Xu, L.; Hu, C.; Huang, T.; Wang, L.; Zhang, J.; Chen, M.; Dong, L.; Jia, Y.; Cai, Y.; Yu, J.; Tang, Y.; Zhang, H.; Ye, Z.; He, P.; Wu, R.; Zhang, C.; Tan, Y.; Xiao, J.; Tao, Y.; Zhu, J.; Xue, J.; Liu, K.; Zhao, C.; Wu, X.; Hu, Z.; Qin, L.; Peng, J.; Li, Z.; Chen, M.; Zhang, X.; Niu, L.; Wang, P.; Wang, Y.; Kuang, H.; Fan, Z.; Zheng, X.; Zhuang, W.; He, Y.; Liu, T.; Yang, Y.; Wang, D.; Liu, Y.; Jiang, J.; Huang, J.; Guo, C. Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation. 2025, Version v1. [Google Scholar]
  396. Zhou, J.; Gao, H.; Voleti, V.; Vasishta, A.; Yao, C.H.; Boss, M.; Torr, P.; Rupprecht, C.; Jampani, V. Stable Virtual Camera: Generative View Synthesis with Diffusion Models; ICCV, 2025; pp. 12405–12414. [Google Scholar]
  397. Zhou, S.; Fan, Z.; Xu, D.; Chang, H.; Chari, P.; Bharadwaj, T.; You, S.; Wang, Z.; Kadambi, A. DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting. ECCV 2024a. [Google Scholar]
  398. Zhou, T.; Tucker, R.; Flynn, J.; Fyffe, G.; Snavely, N. Stereo Magnification: Learning View Synthesis Using Multiplane Images. In ACM Trans Graph; 2018; p. 37. [Google Scholar]
  399. Zhou, Y.; Wang, Q.; Cai, Y.; Yang, H. Allegro: Open the Black Box of Commercial-Level Video Generation Model. arXiv 2024b. [Google Scholar]
  400. Zhu, Y.; Deng, K.; Fauconnier, J.P.; Navarro, I.; Li, D.; Pun, A.; Zhang, Y.; Zhuang, P.; Sun, X.; Agrawala, M.; Bhat, K.; Zhou, T. CubePart: An Open-Vocabulary Part-Controllable 3D Generator. SIGGRAPH Conference Papers, 2026. [Google Scholar]
Figure 2. Taxonomy of 60 representative methods in 20 complementary families. Names link to references; domain and axis labels link to sections. The inset counts 203 catalogued papers by bibliographic year, excluding the Genie 3 blog; 2026 runs through 23 October.
Figure 2. Taxonomy of 60 representative methods in 20 complementary families. Names link to references; domain and axis labels link to sections. The inset counts 203 catalogued papers by bibliographic year, excluding the Genie 3 blog; 2026 runs through 23 October.
Preprints 240396 g002
Figure 3. Visual-generation interfaces: (a) image/video observations with optional history reuse; (b) retained geometry and appearance queried by camera v or modified through an exposed edit unit. Solid arrows carry content/state; dashed arrows carry conditions/queries. All elements are schematic.
Figure 3. Visual-generation interfaces: (a) image/video observations with optional history reuse; (b) retained geometry and appearance queried by camera v or modified through an exposed edit unit. Solid arrows carry content/state; dashed arrows carry conditions/queries. All elements are schematic.
Preprints 240396 g003
Figure 4. Generation relative to use: (a) prepared-content rendering; (b) preview interaction followed by final replacement; (c) generation/decoding during interaction. Solid arrows show processing and feedback; grey dashed arrows show display transfer. Schematics are not to time scale; scene time in (a) is physical time. Images from Bahmani et al.(2024)Bahmani, Skorokhodov, Rong, Wetzstein, Guibas, Wonka, Tulyakov, Park, Tagliasacchi, and Lindell [9], Vachha et al.(2025)Vachha, Kang, Dive, Chidambaram, Gupta, Jun, and Hartmann [302], Xie et al.(2026b)Xie, Sun, Neall, Wu, Cai, and Wetzstein [348].
Figure 4. Generation relative to use: (a) prepared-content rendering; (b) preview interaction followed by final replacement; (c) generation/decoding during interaction. Solid arrows show processing and feedback; grey dashed arrows show display transfer. Schematics are not to time scale; scene time in (a) is physical time. Images from Bahmani et al.(2024)Bahmani, Skorokhodov, Rong, Wetzstein, Guibas, Wonka, Tulyakov, Park, Tagliasacchi, and Lindell [9], Vachha et al.(2025)Vachha, Kang, Dive, Chidambaram, Gupta, Jun, and Hartmann [302], Xie et al.(2026b)Xie, Sun, Neall, Wu, Cai, and Wetzstein [348].
Preprints 240396 g004
Figure 5. XR geometry and timing: (a) coordinate transformations; (b) rectified stereo projection; (c) response latency and display cadence.
Figure 5. XR geometry and timing: (a) coordinate transformations; (b) rectified stereo projection; (c) response latency and display cadence.
Preprints 240396 g005
Figure 6. PartCrafter: (a) part-aware denoising; (b) TripoSG–HoloPart and PartCrafter outputs from a shared RGB image. PartCrafter also takes the requested part count. Images from Lin et al.(2025)Lin, Lin, Pan, Yan, Feng, Mu, and Fragkiadaki [175].
Figure 6. PartCrafter: (a) part-aware denoising; (b) TripoSG–HoloPart and PartCrafter outputs from a shared RGB image. PartCrafter also takes the requested part count. Images from Lin et al.(2025)Lin, Lin, Pan, Yan, Feng, Mu, and Fragkiadaki [175].
Preprints 240396 g006
Figure 7. SceneScape: (a) mesh-guided view synthesis; (b) full model and ablations removing depth tuning, decoder tuning, or mesh retention, labeled (a)–(d). Rows show frames, predicted depth, and COLMAP depth; no shared depth scale or validity key is specified. Images from Fridman et al.(2023)Fridman, Abecasis, Kasten, and Dekel [71].
Figure 7. SceneScape: (a) mesh-guided view synthesis; (b) full model and ablations removing depth tuning, decoder tuning, or mesh retention, labeled (a)–(d). Rows show frames, predicted depth, and COLMAP depth; no shared depth scale or validity key is specified. Images from Fridman et al.(2023)Fridman, Abecasis, Kasten, and Dekel [71].
Preprints 240396 g007
Figure 8. 4D-fy: (a) time-varying field optimization; (b) video-only guidance, no 3D-aware image guidance, no VSD image guidance, image guidance ending after stage II, HexPlane, and full-model results. Within each block, horizontal offsets vary time; vertical offsets vary viewpoint. Instance parameters: θ in (b), φ in (a). Images from Bahmani et al.(2024)Bahmani, Skorokhodov, Rong, Wetzstein, Guibas, Wonka, Tulyakov, Park, Tagliasacchi, and Lindell [9].
Figure 8. 4D-fy: (a) time-varying field optimization; (b) video-only guidance, no 3D-aware image guidance, no VSD image guidance, image guidance ending after stage II, HexPlane, and full-model results. Within each block, horizontal offsets vary time; vertical offsets vary viewpoint. Instance parameters: θ in (b), φ in (a). Images from Bahmani et al.(2024)Bahmani, Skorokhodov, Rong, Wetzstein, Guibas, Wonka, Tulyakov, Park, Tagliasacchi, and Lindell [9].
Preprints 240396 g008
Figure 9. WorldPlay: (a) action conditioning and spatial/temporal history; (b) rotary positional embedding (RoPE) comparison. The stone scene (b1) and statue scene (b2) each pair standard RoPE above with reframed RoPE below, labeled a)/b) internally. Key inputs are W, A, D for b1 and W, A, S for b2. Red boxes mark accumulated errors; right-edge magnifications show geometric inconsistencies. Dataset images from Ling et al.(2024b)Ling, Sheng, Tu, Zhao, Xin, Wan, Yu, Guo, Yu, Lu, Li, Sun, Ashok, Mukherjee, Kang, Kong, Hua, Zhang, Benes, and Bera [177]; generated images from Sun et al.(2026a)Sun, Zhang, Wang, Wu, Wang, Wang, Wang, Zhang, Wang, and Guo [285].
Figure 9. WorldPlay: (a) action conditioning and spatial/temporal history; (b) rotary positional embedding (RoPE) comparison. The stone scene (b1) and statue scene (b2) each pair standard RoPE above with reframed RoPE below, labeled a)/b) internally. Key inputs are W, A, D for b1 and W, A, S for b2. Red boxes mark accumulated errors; right-edge magnifications show geometric inconsistencies. Dataset images from Ling et al.(2024b)Ling, Sheng, Tu, Zhao, Xin, Wan, Yu, Guo, Yu, Lu, Li, Sun, Ashok, Mukherjee, Kang, Kong, Hua, Zhang, Benes, and Bera [177]; generated images from Sun et al.(2026a)Sun, Zhang, Wang, Wu, Wang, Wang, Wang, Zhang, Wang, and Guo [285].
Preprints 240396 g009
Table 2. Datasets, simulators, and evaluation benchmarks used in the review.
Table 2. Datasets, simulators, and evaluation benchmarks used in the review.
Preprints 240396 i008
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.