3D Gaussian Splatting Rasterization: A Survey

1. Introduction

The quest for photorealistic, real-time 3D reconstruction has served as a primary catalyst in computer vision and graphics for decades. While traditional rendering paradigms [1,2] and Neural Radiance Fields (NeRF) [3,4,5,6,7] established rigorous standards for visual fidelity, they remained computationally intractable for interactive applications. This landscape was transformed by 3D Gaussian Splatting (3DGS) [8], which proposed a radical architectural departure: representing scenes explicitly as unstructured collections of millions of anisotropic 3D Gaussians. By leveraging a highly optimized, tile-based differentiable rasterizer [9], 3DGS circumvents expensive neural network queries to achieve strictly real-time frame rates.

The success of 3DGS has rapidly catalyzed a vast and evolving research field. However, reliance on millions of discrete unstructured primitives introduces severe computational bottlenecks. As researchers attempt to scale these representations to unbounded city-level environments, model complex 4D temporal deformations, or deploy them onto resource-constrained edge devices, the original tile-based rasterizer frequently emerges as the primary system bottleneck. Issues such as memory bandwidth saturation, the latency of per-tile visibility sorting, and the representational limitations of strictly symmetric Gaussian kernels necessitate deep algorithmic interventions directly at the rendering pipeline level.

Existing surveys have provided useful overviews of the fast-growing 3DGS field. However, most of them organize recent methods by downstream applications [10,11,12]. This application-centric view is helpful for readers who look for task-specific solutions. Yet it may hide the shared algorithmic problems behind different methods. In explicit 3D reconstruction, many advances are closely related to the rasterization process. For example, methods for sparse-view initialization, anti-aliasing, and geometric densification all rely on similar issues in projection, visibility sorting, and alpha blending.

In this survey, we move away from the common application-driven view and adopt a rasterization-centric perspective on 3D Gaussian Splatting. Instead of treating recent methods as separate solutions for different tasks, we study them through the key stages of the 3DGS rasterization pipeline. This allows us to better reveal the shared algorithmic ideas behind different lines of work.

Our survey differs from prior 3DGS reviews in three main aspects. First, we organize existing methods by where they act in the pipeline, including primitive parameterization, geometric projection, tile assignment, visibility sorting, alpha blending, and gradient propagation. Second, we relate these algorithmic changes to key computational bottlenecks, such as memory bandwidth, tile occupancy, sorting latency, and blending efficiency. This also highlights the importance of hardware–algorithm co-design. Third, we provide cross-method efficiency analyses beyond rendering quality, covering training cost, rendering speed, primitive compactness, and storage compression.

To this end, we propose a fine-grained algorithmic taxonomy that organizes recent literature by their intervention points within the rasterization workflow. Following the establishment of theoretical foundations and point-based pipeline mechanics in Section 2, Section 3 forms the core of our survey by exploring this algorithmic landscape across three hierarchical dimensions:

• Representation and Optimization (Sec. 3.1): Analyzing alternative kernel parameterizations and memory-bounded gradient densification strategies.
• Rasterization Pipeline Mechanics (Sec. 3.2): Dissecting hardware-level thread scheduling, sorting elimination techniques, and the integration of physically-based shading or semantic feature blending.
• Scenario-Driven Rasterization Extensions (Sec. 3.3): Examining how these low-level pipeline innovations translate to overcoming systemic hurdles in 4D dynamic synthesis, massive large-scale extensions, and unposed SLAM scenarios.

Finally, Section 4 synthesizes a comprehensive discussion based on cross-dataset efficiency analyses, followed by an outline of critical open challenges and future research trajectories.

Figure 1. Structural overview and algorithmic taxonomy of our survey. We systematically deconstruct the 3D Gaussian Splatting (3DGS) framework into a fine-grained hierarchy. Following the theoretical foundations, we categorize recent advancements into a three-tiered rasterization taxonomy: Representation and Optimization, Rasterization Pipeline, and Scenario-Driven Rasterization Extensions. Finally, we provide comprehensive cross-dataset evaluations (blue) and outline critical future research directions.

Figure 1. Structural overview and algorithmic taxonomy of our survey. We systematically deconstruct the 3D Gaussian Splatting (3DGS) framework into a fine-grained hierarchy. Following the theoretical foundations, we categorize recent advancements into a three-tiered rasterization taxonomy: Representation and Optimization, Rasterization Pipeline, and Scenario-Driven Rasterization Extensions. Finally, we provide comprehensive cross-dataset evaluations (blue) and outline critical future research directions.

2. Preliminaries: Foundations of 3D Gaussian Splatting

This section reviews the historical evolution of explicit rendering and the core mechanics of the 3DGS differentiable rasterization pipeline.

2.1. From Volume Rendering to Differentiable Splatting

Novel view synthesis is founded on the volume rendering integral [13,14], which NeRF [3] approximates via expensive backward ray tracing. Splatting [1,15] avoids this by forward-mapping primitives and accumulating them in depth order. Point-based graphics [16,17] model complex geometries without meshes. Early systems [18,19,20] used bounding sphere hierarchies but suffered from gaps and aliasing. Surface Splatting [21,22] and EWA Splatting [23,24] introduced anisotropic Gaussian kernels, acting as low-pass filters during projection to eliminate aliasing and handle irregular sampling [25]. Although efficient on GPUs [26], these classical pipelines lacked gradient-based optimization.

Differentiable rendering [27] bridged explicit graphics and deep learning. Mesh-based methods such as OpenDR [28], Soft Rasterizer [29], and neural inverse renderers [30,31] demonstrated 3D property optimization via image-space gradients, while neural rendering [11] explored volumetric implicit representations [32,33,34]. NPBG [35] imbued point primitives with learnable features, replacing traditional shading [36]. Point-NeRF [37], SynSin [38] and Stable View Synthesis [39] combined neural features with point clouds, while Pulsar [40] and ADOP [41] developed efficient sphere-based differentiable rasterizers. However, these methods still relied on CNNs for hole-filling, limiting real-time performance.

A 3DGS scene is typically bootstrapped from a sparse Structure-from-Motion (SfM) point cloud [42,43,44,45], after which its geometric and photometric parameters are jointly refined. 3DGS [8] revitalizes EWA Splatting in a differentiable framework [46,47] for a pure explicit formulation. It allows anisotropic 3D Gaussians to jointly optimize their geometry, opacity and spherical harmonics via gradient descent, achieving state-of-the-art volumetric fidelity at real-time rasterization speed. This architecture continues to inspire specialized hardware accelerators [48], foveated VR rendering [49], and inverse rendering tasks [50,51].

Figure 2. Illustration of the tile-based rasterization rendering process. The image plane is divided into a grid of distinct tiles. When a 3D Gaussian projected onto the 2D screen overlaps multiple tiles, it is duplicated for each intersected tile and assigned a unique sorting key in the format of [Tile ID | Depth]. These duplicated instances are subsequently sorted by depth within each tile to ensure correct visibility ordering. Finally, a thread block is assigned to each tile to perform per-pixel alpha blending, computing the cumulative transmittance and final pixel color where the evaluated opacity of a Gaussian at a specific pixel x is given by ${\alpha }_i=o_i·G_i\left(x\right)$.

Figure 2. Illustration of the tile-based rasterization rendering process. The image plane is divided into a grid of distinct tiles. When a 3D Gaussian projected onto the 2D screen overlaps multiple tiles, it is duplicated for each intersected tile and assigned a unique sorting key in the format of [Tile ID | Depth]. These duplicated instances are subsequently sorted by depth within each tile to ensure correct visibility ordering. Finally, a thread block is assigned to each tile to perform per-pixel alpha blending, computing the cumulative transmittance and final pixel color where the evaluated opacity of a Gaussian at a specific pixel x is given by ${\alpha }_i=o_i·G_i\left(x\right)$.

Table 1. Deconstruction of the 3DGS Differentiable Rasterization Pipeline.

Pipeline Stage	Core Operation	Primary Bottleneck	Related Method Types
Projection	3D covariance to 2D footprint	Aliasing, distortion	Mip-Splatting, 3DGUT
Tile Binning	Duplicate Gaussians to tiles	Memory traffic	Speedy-Splat, FastGS, Taming-3DGS
Sorting	Per-tile depth order	Latency, popping artifacts	StopThePop, PGSR, StochasticSplats
Blending	Alpha accumulation	Transparency, semantic blending	Vol3DGS, LangSplat, 3DGS-Ray-Tracing
Backward Pass	Gradient accumulation	Atomic operations	Mini-Splatting, 2DGS

Table 1. Deconstruction of the 3DGS Differentiable Rasterization Pipeline.

Pipeline Stage	Core Operation	Primary Bottleneck	Related Method Types
Projection	3D covariance to 2D footprint	Aliasing, distortion	Mip-Splatting, 3DGUT
Tile Binning	Duplicate Gaussians to tiles	Memory traffic	Speedy-Splat, FastGS, Taming-3DGS
Sorting	Per-tile depth order	Latency, popping artifacts	StopThePop, PGSR, StochasticSplats
Blending	Alpha accumulation	Transparency, semantic blending	Vol3DGS, LangSplat, 3DGS-Ray-Tracing
Backward Pass	Gradient accumulation	Atomic operations	Mini-Splatting, 2DGS

2.2. 3D Gaussian Representation and Geometric Projection

3DGS parameterizes a scene as an unstructured ensemble of N anisotropic 3D Gaussians. Each primitive is defined by a mean position $\mathit{\mu }\in {\mathbb{R}}^3$, a covariance matrix $\mathbf{\Sigma }$, a scalar opacity $\alpha \in [0,1]$, and Spherical Harmonics (SH) color coefficients $\mathbf{c}$ [8]. The spatial influence of a Gaussian at $\mathbf{x}\in {\mathbb{R}}^3$ is evaluated as:

$$G\left(\mathbf{x}\right)=exp\left(-{\displaystyle \frac{1}{2}}{(\mathbf{x}-\mathit{\mu })}^{\top }{\mathbf{\Sigma }}^{-1}(\mathbf{x}-\mathit{\mu })\right).$$

(1)

To ensure $\mathbf{\Sigma }$ remains positive semi-definite during optimization, it is analytically decomposed into a learnable diagonal scaling matrix $\mathbf{S}$ and a rotation matrix $\mathbf{R}$ (derived from a unit quaternion $\mathbf{q}$): $\mathbf{\Sigma }=\mathbf{R}\mathbf{S}{\mathbf{S}}^{\top }{\mathbf{R}}^{\top }$.

During the forward pass, these 3D primitives are projected onto the 2D image plane. Given a viewing transformation $\mathbf{W}$ and the Jacobian $\mathbf{J}$ of the affine approximation of the perspective projection, the 3D covariance is mapped to screen space as ${\mathbf{\Sigma }}^{\prime }=\mathbf{J}\mathbf{W}\mathbf{\Sigma }{\mathbf{W}}^{\top }{\mathbf{J}}^{\top }$ [23]. To ensure numerical stability and prevent aliasing from sub-pixel primitives, a low-pass filter is applied by augmenting the diagonal: ${\mathbf{\Sigma }}^{″}={\mathbf{\Sigma }}^{\prime }+s\mathbf{I}$. The projected 2D Gaussian weight at pixel $\mathbf{p}$ with 2D mean ${\mathit{\mu }}^{\prime }$ is:

$$G^{\prime }\left(\mathbf{p}\right)=exp\left(-{\displaystyle \frac{1}{2}}{(\mathbf{p}-{\mathit{\mu }}^{\prime })}^{\top }{\left({\mathbf{\Sigma }}^{″}\right)}^{-1}(\mathbf{p}-{\mathit{\mu }}^{\prime })\right).$$

(2)

Standard 3DGS absorbs the normalization constant into the opacity $\alpha$. For rigorous energy conservation during scaling, methods like Mip-Splatting [52] introduce a compensatory factor: $G_{mip}^{\prime }\left(\mathbf{p}\right)=\sqrt{det\left({\mathbf{\Sigma }}^{\prime }\right)/det\left({\mathbf{\Sigma }}^{″}\right)}{G}^{\prime }\left(\mathbf{p}\right)$.

2.3. Tile-Based Rasterization and Differentiable Optimization

To achieve real-time rendering speeds, 3DGS utilizes a highly optimized tile-based rasterizer. The image plane is uniformly partitioned into $16\times 16$ macro-tiles.

Initially, 3D Gaussians are frustum-culled. Surviving primitives are assigned to overlapping screen tiles and sorted by view-space depth using a fast device-side radix sort [53]. For rendering, a CUDA thread block is assigned to each tile. Threads cooperatively fetch sorted Gaussians and accumulate the final pixel color $C\left(\mathbf{p}\right)$ via iterative $\alpha$-blending:

$$C\left(\mathbf{p}\right)={\displaystyle \sum _{i=1}^N}{\mathbf{c}}_i{\alpha }_i{G}_i^{\prime }\left(\mathbf{p}\right){\displaystyle \prod _{j=1}^{i-1}}\left(1-{\alpha }_j{G}_j^{\prime }\left(\mathbf{p}\right)\right).$$

(3)

Thread execution terminates early when the accumulated transmittance drops below a designated threshold.

During the backward pass, gradients propagate from the image-space loss $\mathcal{L}$ through the blending chain. For a Gaussian k, gradients with respect to color ${\mathbf{c}}_k$ and opacity ${\alpha }_k$ are accumulated across all affected pixels using CUDA atomicAdd operations. Simultaneously, screen-space gradients for the 2D mean and covariance are derived via the chain rule on (2), and backpropagated through the viewing transform to explicitly update the 3D position ${\mathit{\mu }}_k$, scaling ${\mathbf{S}}_k$, and rotation ${\mathbf{q}}_k$. This fully differentiable loop enables rapid, end-to-end refinement of the scene geometry and appearance.

Figure 3. The computational graph of the backward pass. The computational graph detailing the forward rendering pipeline and the backpropagation path. This directed acyclic graph illustrates how the final rendering loss L is derived from the learnable parameters of the splats. The primary optimized variables include the 3D means $\mu$, spherical harmonics (SH) coefficients $k_m^l$, scale s, quaternion q, and base opacity o. The graph demonstrates the intermediate transformations required for rendering, such as constructing the 3D covariance matrix $\mathrm{\Sigma }$ from the scale matrix S and rotation matrix R, projecting it to the 2D covariance ${\mathrm{\Sigma }}^{\prime }$ using the Jacobian matrix J, and computing the 2D opacity $\alpha$ and final pixel color C. During training, the gradients flow backward through these explicit differentiable nodes to update the Gaussian parameters.

Figure 3. The computational graph of the backward pass. The computational graph detailing the forward rendering pipeline and the backpropagation path. This directed acyclic graph illustrates how the final rendering loss L is derived from the learnable parameters of the splats. The primary optimized variables include the 3D means $\mu$, spherical harmonics (SH) coefficients $k_m^l$, scale s, quaternion q, and base opacity o. The graph demonstrates the intermediate transformations required for rendering, such as constructing the 3D covariance matrix $\mathrm{\Sigma }$ from the scale matrix S and rotation matrix R, projecting it to the 2D covariance ${\mathrm{\Sigma }}^{\prime }$ using the Jacobian matrix J, and computing the 2D opacity $\alpha$ and final pixel color C. During training, the gradients flow backward through these explicit differentiable nodes to update the Gaussian parameters.

3. Taxonomy of 3DGS Rasterization Algorithms

We now present a structured algorithmic taxonomy of the 3DGS research landscape. Rather than categorizing by end-use application, we provide a viewpoint based on the specific rasterization and optimization sub-problems they address.

3.1. Representation and Optimization

3.1.1. Kernel Parameterizations and Geometry Projection

Alternative Geometric Distributions

The elliptically anisotropic 3D Gaussian serves as the fundamental primitive for standard 3DGS. However, its ellipsoidal symmetry, infinite support, and soft exponential decay often precipitate artifacts when modeling sharp edges or topologically complex boundaries. As shown in Figure 4, this inherent limitation has catalyzed a rich landscape of alternative kernel representations and projection strategies. GES [54] replaces the Gaussian kernel with a generalized exponential function that provides tighter support and sharper falloff. The kernel is defined as

$$G_{\mathrm{GES}}\left(\mathbf{p}\right)=exp\left(-{\displaystyle \frac{1}{2}}{\left({(\mathbf{p}-\mathit{\mu })}^T{\mathbf{\Sigma }}^{-1}(\mathbf{p}-\mathit{\mu })\right)}^{\beta }\right),$$

(4)

where the shape parameter $\beta >0$ controls the decay rate. For $\beta =1$, this recovers the standard Gaussian. 3DHGS [55] models each primitive as a truncated Gaussian:

$$G_{\mathrm{half}}\left(\mathbf{p}\right)=exp\left(-{\displaystyle \frac{1}{2}}{(\mathbf{p}-\mathit{\mu })}^T{\mathbf{\Sigma }}^{-1}(\mathbf{p}-\mathit{\mu })\right)·{\mathbf{1}}_{{\mathbf{n}}^T(\mathbf{p}-\mathit{\mu })\ge 0},$$

(5)

where $\mathbf{n}$ is a learnable normal direction. The hard cut introduces a true discontinuity, allowing the method to capture razor-sharp edges without blurring. 3D Student Splatting [56] generalizes the Gaussian to a Student’s t-distribution, whose heavy tails make it robust to outliers and offer more flexible shape control through the degrees of freedom. Moving beyond ellipsoids, 3D Convex Splatting [57] represents each primitive as a smooth convex polyhedron defined by the intersection of K half-spaces, allowing compact representation of complex geometries with sharp creases. Similarly, Deformable Beta Splatting uses a beta distribution density, $G_{\beta }\left(\mathbf{p}\right)\propto {\left({\mathbf{p}}^T\mathbf{A}\mathbf{p}\right)}^{a-1}{(1-{\mathbf{p}}^T\mathbf{A}\mathbf{p})}^{b-1}$, to enable richer deformation fields that can bend and stretch while maintaining a closed-form projection [58]. QuadBox [59] replaces the Gaussian with a geometry-aware bounding box, where the influence is defined by the smoothed distance to the box surface $G_{\mathrm{box}}\left(\mathbf{p}\right)=exp\left(-\lambda ·\mathrm{sdist}(\mathbf{p};\mathbf{B})\right),$ which accelerates rasterization as box-ray intersections are cheaper than Gaussian evaluations. N-Dimensional Gaussians lift the representation to a generic D-dimensional space, enabling joint modeling of appearance, geometry, and other attributes with a single kernel:

$$G_D\left(\mathbf{x}\right)=exp\left(-{\displaystyle \frac{1}{2}}{(\mathbf{x}-\mathit{\mu })}^T{\mathbf{\Sigma }}_D^{-1}(\mathbf{x}-\mathit{\mu })\right),\mathbf{x}\in {\mathbb{R}}^D,$$

(6)

with the projection onto the image plane defined by marginalizing the extra dimensions [60]. 3D Skew Gaussian Splatting [61] employs a skew-normal distribution to better represent asymmetrical scattering and complex radiance fields:

$$G_{\mathrm{skew}}\left(\mathbf{p}\right)=2\mathrm{\Phi }\left({\mathit{\alpha }}^T{\mathbf{L}}^{-1}(\mathbf{p}-\mathit{\mu })\right)·\mathcal{N}(\mathbf{p};\mathit{\mu },\mathbf{\Sigma }),$$

(7)

where $\mathrm{\Phi }$ is the CDF of normal distribution, $\mathbf{L}$ is the Cholesky factor, and the skew vector $\mathit{\alpha }$ adds directional asymmetry.

Surface and Planar Approximations

To better approximate surfaces, several works collapse one dimension of the 3D Gaussian, effectively turning it into a 2D surfel. SUGAR [62] explicitly aligns Gaussian primitives with a mesh surface by adding a regularization term that encourages the shortest-axis of the covariance to align with the surface normal. Gaussian Opacity Fields (GOF) [63] proposes a paradigm shift: instead of densifying the Gaussian cloud to fill volumes, it directly models a continuous opacity field from the Gaussians and extracts a surface at the level set $\alpha \left(\mathbf{x}\right)=0.5$. The opacity field is defined as a smooth approximation of the maximum contribution from all Gaussians:

$$\alpha \left(\mathbf{x}\right)=1-{\displaystyle \prod _{i=1}^N}\left(1-{\alpha }_i{G}_i\left(\mathbf{x}\right)\right).$$

(8)

This formulation enables efficient adaptive surface reconstruction in unbounded scenes, dramatically reducing the number of required primitives. 2D Gaussian Splatting (2DGS) replaces the 3D covariance with a planar one, ${\mathbf{\Sigma }}_{2\mathrm{D}}=\mathbf{R}\mathrm{diag}(s_1,s_2,0){\mathbf{R}}^T$, which concentrates all density on a disk. This enables a geometrically consistent extraction of depth and normal via the point-to-plane distance:

$$d\left(\mathbf{p}\right)={\displaystyle \frac{{(\mathbf{p}-\mathit{\mu })}^T\mathbf{R}{\mathbf{e}}_3}{\parallel \mathbf{R}{\mathbf{e}}_3\parallel }},$$

(9)

greatly improving surface alignment [64]. As shown in Figure 8, PGSR employs planar Gaussians that simplify the projection and rasterization steps, leading to more regular memory access patterns and higher efficiency on existing hardware [65]. UniGS [66] proposes a unified geometry-aware framework that jointly optimizes the Gaussian parameters and a signed distance field, using the SDF as an additional regularizer to improve densification and reconstruction accuracy across multiple modalities. Gabor Splatting [67] uses a Gabor wavelet, which is a product of a Gaussian and a sinusoid and provides a stronger response to oriented textures and high-frequency surface details.

Anti-Aliasing and Generalized Projections

Point-sampled evaluation of the Gaussian footprint triggers severe high-frequency aliasing during extreme zoom-outs. Mip-Splatting [52] mitigates aliasing by constraining the 3D frequency content of each Gaussian. It enforces that the largest scale $s_{max}$ of the covariance satisfies

$$s_{max}\ge {\displaystyle \frac{\Delta p}{2\sqrt{2log2}}}f,$$

(10)

where $\Delta p$ is the pixel footprint and f is a frequency factor, effectively applying a 3D low-pass filter. Analytic-Splatting [68] pushes this further by replacing the point sample with an approximate integral of the 2D Gaussian over the pixel square $P_{uv}$. 3DGUT [69] generalizes the projection model to handle distorted cameras and secondary rays. Instead of a pinhole projection ${\mathit{\mu }}^{\prime }=\mathbf{P}\left(\mathit{\mu }\right)$, it defines a per-pixel ray function $\mathbf{r}(u,v)$ and samples the Gaussian along that ray, thereby making the rendering compatible with non-conventional optics. ODGS [70] reconstructs scenes from omnidirectional images by adjusting the projection to a spherical camera model and using feature blending that accounts for the 360° field of view. Multi-Scale 3DGS [71] addresses aliasing in unbounded scenes by building a mip-map pyramid of Gaussians. At rendering time, the appropriate level-of-detail l is selected based on the projected pixel size, and the blending function becomes a combination of scales. Multi-Sample Anti-Aliasing directly supersamples each pixel and averages the contributions [72], while Efficient Differentiable Hardware Rasterization designs dedicated GPU circuits that perform analytic anti-aliasing on the fly [73]. Physical adversarial camouflage [74] generation via 3DGS leverages the differentiability of the projection to perturb the color and texture of Gaussians, solving a min-max optimization over the rendered views under varying viewpoints.

Overall, these kernel-level designs trade the simplicity and hardware efficiency of standard Gaussian splats for greater geometric expressiveness. Different kernel designs can improve robustness and asymmetry modeling, but often require more complex projection and integration in the rasterization pipeline.

Figure 5. Effect of collapsing 3D Gaussians into 2D splats and 3DGS’s depth simplification: (a) Integrating Gaussians along view rays r requires careful consideration of potentially overlapping 1D Gaussians. (b) Using flattened 2D splats and view-space z as depth (projection of $\mu$ onto v) puts 2D splats on spherical segments around the camera, inverting the relative positions of the two Gaussians along the example view ray. (c) Camera rotation inverts the order along r, resulting in popping. (d) Camera translation does not alter the distance compared to (b).

Figure 5. Effect of collapsing 3D Gaussians into 2D splats and 3DGS’s depth simplification: (a) Integrating Gaussians along view rays r requires careful consideration of potentially overlapping 1D Gaussians. (b) Using flattened 2D splats and view-space z as depth (projection of $\mu$ onto v) puts 2D splats on spherical segments around the camera, inverting the relative positions of the two Gaussians along the example view ray. (c) Camera rotation inverts the order along r, resulting in popping. (d) Camera translation does not alter the distance compared to (b).

3.1.2. Densification, Regularization and Compactness

The structural evolution of a 3DGS scene relies on a coupled loop of forward rendering and adaptive density control. Standard discrete topological changes are susceptible to exploding primitives and massive memory footprints. Recent research refines these dynamics to enforce stability and extreme compactness.

Probabilistic and Perceptual Densification

The original densification heuristic relies on the noisy magnitude of positional gradients. 3DGS-MCMC [75] reinterprets the Gaussian cloud as a set of samples from an underlying probability distribution and uses Markov Chain Monte Carlo sampling to guide the life cycle of primitives. Specifically, it proposes a Metropolis–Hastings step where a new Gaussian is spawned at position ${\mathit{\mu }}_{\mathrm{new}}$ with probability proportional to the rendering error:

$${\alpha }_{\mathrm{spawn}}=min\left(1,{\displaystyle \frac{\mathcal{L}\left({\mathit{\mu }}_{\mathrm{new}}\right)}{\mathcal{L}\left({\mathit{\mu }}_{\mathrm{old}}\right)}}·{\displaystyle \frac{q\left({\mathit{\mu }}_{\mathrm{old}}\right|{\mathit{\mu }}_{\mathrm{new}})}{q\left({\mathit{\mu }}_{\mathrm{new}}\right|{\mathit{\mu }}_{\mathrm{old}})}}\right),$$

(11)

and prunes Gaussians with low expected contribution according to a Markov chain equilibrium. This probabilistic framework helps to explore regions with unknown gradients during training, yielding more efficient and artifact-free clouds. Perceptual-GS [76] replaces the simple gradient threshold with a scene-adaptive perceptual signal. It computes a local perceptual error map using a lightweight vision-transformer head and densifies only in regions where SSIM falls below a dynamic threshold, significantly improving detail in visually salient areas. GaussianPro [77] introduces a progressive propagation strategy that expands the Gaussian cloud in a fronto-parallel manner, ensuring smooth expansion and better coverage.

Regularization and Artifact Reduction

To prevent overfitting and high-frequency artifacts during training, FreGS [78] applies progressive frequency regularization to the primitives, while Faster-GS [79] finds that gradient updates for the scaling and rotation parameters often oscillate due to a poorly conditioned learning rate ratio. They address this by rescaling the scaling gradient with the inverse of the current scale, which stabilizes convergence and cuts training time by nearly half. ABSGS [80] tackles the problem of fine detail recovery that is smoothed out during optimization. The reconstructed image is compared against the ground truth with a high-pass filter, and only the high-frequency residuals are back-propagated, effectively sharpening edges and thin structures.

Sparsification and Model Compression

To explicitly bound memory footprints and accelerate training, recent methods aggressively prune redundancy. Mini-Splatting [81] explicitly constrains the total number of Gaussians by imposing a hard limit $N_{max}$ and formulating the optimization as a mixed integer problem. It iteratively prunes the least important Gaussians and splits the most important ones, keeping the total count constant. Taming 3DGS [82] explores a spectrum of resource-efficiency trade-offs, from predictable schedule to adaptive importance guidance as shown in Figure 9, enabling high-quality rendering on consumer-grade GPUs. LightGaussian [83] compresses the Gaussians by factorizing the SH parameter and applying a structured pruning strategy, achieving over 200 FPS with slight quality loss. RTGS [84] applies a multi-level redundancy reduction strategy that removes Gaussians based on both spatial overlap and temporal consistency. Speedy-Splat [85] accelerates both training and rendering by simultaneously applying pixel-sparse sampling and primitive-sparse selection as shown in Figure 6. GaussianSPA [86] proposes an optimizing-sparsifying loop that interleaves standard training with a sparsification operator, removing redundant primitives while boosting the influence of the remaining collection. Shorter Gaussian Lists [87] investigates training from a tiny initial set of Gaussians, showing that careful initialization and regularization help achieve comparable quality and accelerated pipeline. FastGS [88] re-engineers the entire training loop to finish optimization. It employs a dynamic sparse rasterization that skips tiles with negligible transmittance, combined with a fast gradient checkpointing scheme that recomputes only the necessary intermediate values. MMGS [89] treats the compression of 3D Gaussian as an optimal transport problem, learning a compact set of summary Gaussians that best represent the original cloud under multi-view ranking constraints $C_i={\sum }_{v\in \mathcal{V}}{\sum }_{u\in \mathcal{I}}{\mathbf{M}}_u^{\left(v\right)}·\mathbb{I}\left({\omega }_{i,u}^{\left(v\right)}>{ϵ}_{\mathrm{v}}\right).$ The optimization minimizes the Wasserstein distance between the original and compressed sets:

$${\mathbf{C}}_{ij}=d_G^2({\mathcal{S}}_{src},{\mathcal{S}}_{tgt})={\parallel {\mu }_i-{\mu }_j\parallel }_2^2+{\parallel {\mathrm{\Sigma }}_i^{1/2}-{\mathrm{\Sigma }}_j^{1/2}\parallel }_F^2,$$

(12)

maintaining rendering quality with only 10% primitives and 10× accelerated training speeds compared to 3DGS.

Feed-Forward and Initialization Strategies

Finally, a distinct subset of methods completely bypasses or drastically shortens the per-scene gradient descent loop. MVSplat [90] trains a feed-forward deep neural network to directly regress 3DGS parameters from sparse multi-view images in a single pass. Alternatively, RadSplat [91] leverages the robust initialization of NeRF to generate a highly accurate dense point cloud, which is subsequently converted into Gaussians. This hybrid approach circumvents the topological pitfalls of raw SfM initialization, achieving unprecedented 900+ FPS rendering with high quality.

In summary, densification methods improve coverage and high-frequency details, whereas pruning and compression methods reduce memory and training cost. The central challenge is to balance adaptive primitive growth with stable optimization and bounded resource consumption.

3.2. Rasterization Pipeline

With the underlying representations optimized, the computational bottleneck shifts to the execution logic on the GPU. This section dissects the rasterization pipeline, detailing innovations in physically-based shading, semantic feature blending, and low-level hardware thread scheduling.

3.2.1. Alpha Blending, Feature Fields and Shading

Physically-Based Shading and Relighting

The alpha blending algorithm in the rendering stage is not only a compositing step but also a framework that can be redesigned to incorporate complex light transport or semantic features, enabling physically based shading, open-vocabulary segmentation, and even interactive question answering. The original 3DGS uses a view-dependent spherical harmonics color model, which fails to represent specular reflections, glossy materials, and global illumination. A major paradigm shift is embedding physically-based material properties into each Gaussian for deferred shading. Relightable 3DGS [92] decomposes the outgoing radiance into a BRDF integration that respects material parameters. For a given view direction ${\mathit{\omega }}_o$ and an incident lighting direction ${\mathit{\omega }}_i$, the reflected radiance is computed using the microfacet Cook-Torrance BRDF:

$$f_s({\mathit{\omega }}_o,{\mathit{\omega }}_i)={\displaystyle \frac{D(\mathit{h};r)·F({\mathit{\omega }}_o,\mathit{h})·G({\mathit{\omega }}_i,{\mathit{\omega }}_o,h;r)}{(\mathit{n}·{\mathit{\omega }}_i)·(\mathit{n}·{\mathit{\omega }}_o)}},$$

(13)

where $\mathit{h}=({\mathit{\omega }}_i+{\mathit{\omega }}_o)/2$ is the half vector, $D,F$ and G denote the normal distribution function, Fresnel term and geometry term. The PBR color of each Gaussian is then given by:

$$c^{\prime }\left({\omega }_o\right)={\displaystyle \sum _{i=0}^{N_s}}(f_d+f_s({\omega }_o,{\omega }_i))L_i\left({\omega }_i\right)({\omega }_i·\mathit{n})\Delta {\mathit{\omega }}_i,$$

where ${\omega }_i$ denotes the solid angle associated with the sampled incident direction. To compute direct illumination from a point source, the method traces secondary rays from each Gaussian to the light, evaluating a shadow map. This enables relighting the entire scene under novel illumination conditions while preserving real-time performance during view synthesis. GaussianShader [93] takes a similar approach but simplifies the BRDF to a neural shader that predicts specular and diffuse components directly from the view direction and a learnable material feature. GS-IR [94] performs inverse rendering by jointly optimizing the Gaussian geometry and an environment light probe, using the differentiable rasterizer to backpropagate gradients through the rendering equation. Geo-Splatting [95] guides the inverse rendering process with explicit geometric priors, adding a normal consistency loss between the Gaussian normals and a monocular depth estimator, which improves material decomposition.

Volumetric Consistency and Textures

Standard 3DGS blending treats each Gaussian as an independent emitter, which violates physical absorption-emission models for heterogeneous transparent media and implausible transparency effects. Don’t Splat your Gaussians [96] introduce volumetric ray-traced primitives with closed-form transmittance for efficient media rendering. 3d gaussian ray tracing [97] uses hardware-accelerated ray tracing via BVH and bounding meshes for semi-transparent particles. As shown in Figure 7, Vol3DGS [98] enforces that the alpha blending equation exactly satisfies the front-to-back absorption-emission model. It derives the discrete compositing as an unbiased Monte Carlo estimator of the continuous integral:

$$C\left(p\right)={\int }_{t_{min}}^{t_{max}}\mathbf{c}\left(t\right)\tau \left(t\right)dt,\tau \left(t\right)=exp\left(-{\int }_{t_{min}}^t\sigma \left(s\right)ds\right),$$

(14)

By introducing a continuous density field along the ray, the method resolves artifacts in scenes with semi-transparent objects. The necessity of volumetric accuracy is discussed by [99], who find that for purely opaque scenes the approximation is sufficient, but transparent and heterogeneous media benefit from the correction. Textured Gaussians [100] abandons SH coefficients in favor of explicit 2D textures mapped onto each primitive. The color of a Gaussian is sampled from the texture according to its local UV coordinates, enabling high-frequency appearance details without increasing the number of primitives. Neural Texture Splatting [101] further encodes textures as small neural networks evaluated at the Gaussian coordinates, which can also be deformed to model dynamic appearances and fine geometric displacements.

Semantic and Vision-Language Features

The speed of the 3DGS rasterizer makes it an ideal platform for distilling 2D vision-model features into a 3D representation. Feature-3DGS [102] and LangSplat [103] pioneer this direction by adding a high-dimensional feature vector ${\mathbf{f}}_i$ to each Gaussian, which is used to render a feature map through alpha-blending:

$$F\left(p\right)=\sum _i{\mathbf{f}}_i{\alpha }_i{\displaystyle \prod _{j=1}^{i-1}}(1-{\alpha }_j).$$

(15)

The resulting 2D feature map can then be decoded by a segmentation head or compared against text embeddings from CLIP, enabling open-vocabulary semantic segmentation in real time. Language Embedded 3D Gaussians [104] expands this to full scene understanding, encoding not only pixel-level features but also object-level embeddings for phrase grounding. SplatTalk [105] and 3D Vision-Language GS [106] take the feature blending further by projecting vision-language models into the 3DGS framework. They train a joint embedding space where the rendered feature map can be queried with natural language, enabling tasks like 3D visual question answering and zero-shot semantic reasoning. NG-GS [107] uses a pretrained NeRF as a teacher to guide the feature distillation, improving the alignment between geometry and semantics. Fast-SegSim [108] adapts the feature rendering pipeline for robotics simulation, achieving real-time open-vocabulary segmentation by fusing 3DGS with a lightweight 2D segmentation network. LeafFit [109] applies 3DGS to create digital plant assets, leveraging the alpha blending and feature fields to accurately reconstruct translucent leaf structures. SeqAffordSplat [110] performs sequential affordance reasoning on the 3DGS representation, where the rendered feature fields encode interaction probabilities over time, allowing robots to plan actions in the rendered 3D scene.

3.2.2. Hardware Thread Scheduling, Visibility and Sorting

Sorting Elimination and Hierarchical Binning

The tile-based rasterizer of 3DGS achieves remarkable speed through a tight co-design with GPU hardware, but it also introduces bottlenecks in per-tile depth sorting, workload imbalance, and memory access. Research in this domain addresses these low-level mechanisms directly, often by reformulating the visibility problem or redesigning the rendering schedule. The per-tile radix sort is a major contributor to rendering latency. When the sorting results are retained, its efficiency becomes paramount. StochasticSplats [111] removes this step by applying stochastic transparency: each Gaussian is probabilistically accepted or discarded along the ray according to a random decision, producing a sort-free compositing that is hardware-friendly. AAA-Gaussians [112] eliminates sorting analytically by pre-integrating the alpha blending equation; it computes an equivalent order-independent transparency through a rational polynomial approximation of the visibility term, thereby suppressing popping artifacts without per-pixel depth sorting. StopThePop [113] observes that a global sort per tile often leads to view-inconsistent popping when Gaussian depths change across frames. As shown in Figure 5, it proposes a hierarchical sort that first groups Gaussians into coarse depth bins and then sorts only within the bin that covers the depth discontinuities. DistWar [114] accelerates sorting by exploiting the hardware accelerated visibility ordering in the rasterizer to perform a fast depth test and bypass the CUDA radix sort.

Hardware-Algorithm Co-Design

To maximize GPU utilization, rendering logic is increasingly co-designed with hardware primitives. GS-TG [115] proposes a tile-grouping architecture that merges adjacent tiles into larger work units, reducing redundant sorting and improving data reuse within GPU streaming multiprocessors. GauRast [116] augments standard triangle rasterizers with dedicated units for Gaussian projection and blending, achieving an order of magnitude speedup in the rasterization stage. GSAcc [117] introduces a depth-speculation mechanism that predicts per-tile depth ranges, enabling early culling of occluded Gaussians before sorting. GEMM-GS [118] maps the alpha-blending and sorting operations to a sequence of General Matrix Multiplications (GEMMs). The per-tile blending becomes a matrix multiplication $\mathbf{C}=\mathbf{A}\mathbf{B}$ where $\mathbf{A}$ contains the Gaussian attributes and $\mathbf{B}$ the pixel-wise accumulation weights.

Deployment Architectures and Edge Rendering

To support constrained environments, pipeline execution can be re-engineered. gsplat [119] continuously integrates low-level scheduling optimizations into a unified high-performance API. FlashGS [120] focuses on accelerating the forward process by partitioning the screen into finer tiles and scheduling them on a multi-GPU pipeline. Mobile-GS [121] brings real-time Gaussian splatting to smartphones for resource-constrained deployments by adopting a memory-centric rendering design. It loads only the Gaussians visible in the current view frustum from flash storage, uses a quantized integer representation for positions and colors, and performs the blending in a lower-precision shader.

Figure 8. Unbiased depth rendering. (a) Illustration of the rendered depth: We take a single Gaussian, flatten it into a plane, and fit it onto the surface as an example. PGSR’s rendered depth is the intersection point of rays and surfaces, matching the actual surface. In contrast, the depth from previous methods corresponds to a curved surface and may deviate from the actual surface. (b) PGSR uses true depth to supervise two different depth rendering methods. After optimization, PGSR maps the positions of all Gaussian points. Gaussians of our method fit well onto the actual surface, while the previous method results in noise and poor adherence to the surface.

Figure 8. Unbiased depth rendering. (a) Illustration of the rendered depth: We take a single Gaussian, flatten it into a plane, and fit it onto the surface as an example. PGSR’s rendered depth is the intersection point of rays and surfaces, matching the actual surface. In contrast, the depth from previous methods corresponds to a curved surface and may deviate from the actual surface. (b) PGSR uses true depth to supervise two different depth rendering methods. After optimization, PGSR maps the positions of all Gaussian points. Gaussians of our method fit well onto the actual surface, while the previous method results in noise and poor adherence to the surface.

3.3. Scenario-Driven Rasterization Extensions

3.3.1. 4D Dynamics and Temporal Deformations

Deformation Fields and Splines

Dynamic 3DGS extends rasterization from a static primitive set to a time-varying primitive field. The key rasterization challenge is no longer only projecting and sorting Gaussians, but doing so under temporally varying geometry, deformation fields, and visibility orders, often via a deformation network that predicts per-frame adjustments to a canonical Gaussian cloud [6,7]. A common paradigm is to keep a canonical 3DGS model at time $t_0$ and learn a deformation field $F_{\theta }$ that, for each timestep t, predicts a displacement for every Gaussian. Deformable 3D Gaussians [122] implements this with an MLP that takes positional and temporal encodings, directly supervising with multi-view videos. 4D Gaussian Splatting [123] extends this to real-time rendering by decomposing the 4D representation into a set of time-varying 3D Gaussians, where the deformation network is replaced by a compact hex-plane encoder. SplineGS replaces the discrete per-frame deformation with a smooth spline that models the trajectory of each primitive’s center and rotation as a function of time:

$${\mathit{\mu }}_i\left(t\right)={\displaystyle \sum _{k=0}^K}{\mathbf{B}}_k\left(t\right){\mathbf{p}}_{i,k},$$

(16)

where ${\mathbf{B}}_k$ are B-spline basis functions and ${\mathbf{p}}_{i,k}$ are control points. This ensures physically plausible, smooth motion with fewer parameters [124]. TreeSplat organizes the deformation as a mergeable tree, where each node represents a rigid part of an articulated object, and the overall deformation is computed by traversing the kinematic chain [125].

Flow, Tracking, and Control

Gaussian-Flow treats each Gaussian as a dynamic particle advected by a 3D velocity field, and the 4D reconstruction is obtained by integrating the flow over time [126]. Spacetime adds a temporal dimension to the feature field, enabling real-time interpolation of both color and geometry between keyframes [127]. Gaussian Splatting Lucas-Kanade directly applies optical flow estimation to the rendered Gaussians, using the flow to align dynamic objects across frames and improve temporal consistency [128]. Real-time 4DGS achieves interactive frame rates by building a specialized renderer that fuses static background Gaussians with a small, fast-updating set of dynamic foreground Gaussians [129]. Dynamic 3D Gaussians performs tracking by persistent dynamic view synthesis, linking the motion of each Gaussian to the movement of a corresponding 3D point in the scene [130]. SC-GS allows sparse user-provided control points to edit the dynamic scene, propagating edits across time via a learned deformation prior [131]. SpeeDe3DGS [132] accelerates deformable 3DGS by applying temporal pruning. They identify static and active Gaussians and group the dynamic parts by motion similarity to reduce the number of deformation queries. RetimeGS [133] achieves continuous-time 4D reconstruction by modeling the trajectories with a neural ODE solver, allowing frame rates to be arbitrarily adjusted at test time. Gaufre [134] introduces a temporal coherence heuristic into the tile sorting, avoiding re-sorting of Gaussians whose depth order has not changed between frames.

3.3.2. Large-Scale and Scalable Formulations

Large-scale scenes stress the rasterizer through primitive count, memory bandwidth, tile occupancy, and out-of-core data movement. Overcoming this to enable city-scale reconstruction is a major challenge, addressed via spatial partitioning with level-of-detail (LoD) and out-of-core distributed training. Earlier lessons from scaling NeRF are provided by Block-NeRF [135] and Mega-NeRF [136], which introduced spatial partitioning for neural radiance fields. MatrixCity offers a large-scale city dataset specifically designed for benchmarking scalable neural rendering methods [137].

Block-Based Partitioning

A straightforward approach to massive scenes is divide-and-conquer. VastGaussian and CityGaussian [138,139] partition large scenes into overlapping blocks, train independent 3DGS models per block, and merge them at rendering time. CityGS-X [140] extends this with a geometry-aware architecture that refines block boundaries according to building footprints, achieving highly accurate urban reconstruction.

Hierarchical and LoD Structures

To avoid rendering millions of far-away, sub-pixel Gaussians, multi-resolution structures are employed. Scaffold-GS [141] anchors scene content to a sparse voxel grid, using the grid as a scaffold to allocate Gaussians only in occupied regions, which naturally adapts to view distance and improves memory efficiency. Octree-GS [142] builds an octree over the scene, associating each node with a merged Gaussian that summarizes the radiance and opacity of its children. Hierarchical 3D Gaussian Representations [143] render very large datasets in real time by storing a multi-resolution pyramid of Gaussians. Google’s LODGE [144] introduces a level-of-detail framework for large-scale 3D Gaussian Splatting, enabling efficient rendering by dynamically selecting appropriate detail levels. Similarly, Nvidia’s HiGS [145] proposes a hierarchical rendering architecture, achieving real-time performance through adaptive level selection and efficient GPU execution. FilterGS [146] proposes a traversal-free, parallel filtering structure that selects a single scale for each Gaussian via a learned network, avoiding expensive tree traversal at render time.

Urban Scenes and Autonomous Driving

Applying 3DGS to cityscapes and street views requires handling complex semantics and dynamic objects. UrbanGS [147] targets large-scale urban scenes, combining semantic segmentation to allocate Gaussians only on building facades and road surfaces. Proxy-GS [148] uses a coarse proxy mesh to guide Gaussian placement, significantly reducing the number of Gaussians required for background regions. Generative Gaussian Splatting [149] generates unbounded 3D cities by progressively extending the Gaussian cloud outward from a given seed region with a diffusion prior. For driving scenes, Street Gaussians [150] model dynamic urban environments by associating each vehicle or pedestrian with a separate 3DGS sub-model that can be moved independently. DrivingGaussian [151] composites surrounding views from multiple cameras into a single dynamic scene, aligning the Gaussian clouds of different vehicles with their tracked trajectories.

Out-of-Core and Distributed Optimization

Grendel-GS partitions the Gaussian list across multiple GPUs, where each GPU renders its own subset of Gaussians and combines the results via inter-GPU communication at partition boundaries [152]. CLM removes the GPU memory barrier by moving Gaussian states between CPU and GPU memory during optimization, using a smart caching policy to keep the most frequently updated Gaussians on the GPU [153]. TideGS pushes the frontier to over one billion primitives by implementing a fully out-of-core optimization that streams Gaussians from disk in a view-dependent order, training on a single machine with only a fraction of the data in GPU memory [154]. Wavelet-GS compresses the SH coefficients of each Gaussian using a wavelet decomposition, discarding high-frequency components that are not visible at typical viewing distances [155]. DWTGS rethinks frequency regularization for sparse-view 3DGS, applying a discrete wavelet transform on the rendered image and penalizing the loss in the wavelet domain to encourage sharper edges [156]. Momentum-GS leverages momentum self-distillation, where a teacher model (an exponential moving average of the parameters) guides the optimization of the student, improving convergence quality for large scenes [157].

Compression via Quantization and Codebooks

Concurrently, Compressed3D [158] and Compact3D [159] introduce vector quantization and codebooks to cluster Gaussian parameters with similar values into shared codewords for more efficient storage. Building on anchor-based representation, Context-GS [160] and CompGS [161] adopt context-aware approaches that explicitly model hierarchical relationships among anchors to enhance representational capacity. Pushing this further, HAC++ [162] incorporates an interpolated hash grid to provide spatial priors for unorganized anchors, establishing a sophisticated context model for arithmetic entropy coding. Alongside academic advances, industrial-grade data formats have emerged to standardize 3DGS delivery. For example, the LCC format family has evolved from LCC11 to LCC22. The original LCC1 format already supports spatial chunking and Level-of-Detail (LoD) organization for progressive loading and large-scale scene visualization. LCC2 further refines this design by decoupling data storage from LoD partitioning, organizing scene contents through a flexible hierarchical LoD tree that can index different payload types and storage backends. Together with compact binary layout and parameter quantization, such formats can reduce storage by up to 90% compared with standard PLY files, facilitating out-of-core streaming and interactive rendering for massive scenes on WebGPU clients.

Figure 9. Overview of Taming 3DGS method. (a) They propose a systematic redesign of 3DGS densification. To select Gaussians to densify, they sample training views and compute per-pixel saliency. A scoring function F combines gradient, saliency, and primitive properties into a per-Gaussian score. (b) The addition of new Gaussians follows a predictable schedule. They follow a growth curve that mimics 3DGS’ behavior and can be fitted to yield any desired model size after training.

Figure 9. Overview of Taming 3DGS method. (a) They propose a systematic redesign of 3DGS densification. To select Gaussians to densify, they sample training views and compute per-pixel saliency. A scoring function F combines gradient, saliency, and primitive properties into a per-Gaussian score. (b) The addition of new Gaussians follows a predictable schedule. They follow a growth curve that mimics 3DGS’ behavior and can be fitted to yield any desired model size after training.

3.3.3. Sparse-View, Unposed Scenarios and 3DGS-SLAM

Unposed Reconstruction and Feature Matching

Simultaneous Localization and Mapping (SLAM) with 3DGS is a crucial online application that unifies camera tracking and scene reconstruction. In this setting, the differentiable rasterizer provides a direct link from camera pose parameters to photometric error, enabling joint optimization. Many techniques for pose-free and sparse-view reconstruction build on earlier NeRF-based approaches [163,164,165,166,167,168,169]. For scenarios with very few input views with unknown camera poses, the 3DGS optimization problem becomes severely ill-posed. FSGS enables real-time few-shot view synthesis by initializing the Gaussian cloud from a learned prior over a shape category, then fine-tuning with a few images [170]. COLMAP-free 3DGS eliminates the need for external SfM by jointly optimizing camera poses and Gaussian parameters from scratch, using a progressive seeding strategy that starts with a single Gaussian and grows the cloud as more poses are discovered [171]. Depth-regularized Optimization improves the quality in few-shot settings by adding a monocular depth prior as an additional supervision signal for the Gaussian positions [172]. DropoutGS [173] introduces a dropout-like mechanism into the 3DGS training loop. During each forward pass, a random subset of Gaussians is temporarily deactivated, forcing the remaining primitives to compensate and learn a more generalizable representation and substantially improves novel view synthesis in sparse-view scenarios. NexusGS further exploits epipolar geometry: it computes epipolar depth priors from a small set of sparse matches and uses them to regularize the Gaussian mean and covariance during optimization [174]. Pseudo-View Enhancement via Confidence Fusion addresses unposed sparse-view reconstruction by generating pseudo-views from an ensemble of depth-to-3D estimators and fusing them based on their confidence scores, providing a richer supervision signal [175].

3DGS-based SLAM Systems

In SLAM and unposed reconstruction, rasterization becomes part of an online optimization loop, where camera poses, Gaussian parameters, and rendering residuals must be jointly updated under tight latency constraints. MonoGS [176] pioneered the tight coupling of a 3DGS map with a camera tracking frontend. It maintains a keyframe-based Gaussian cloud and, for each incoming frame, optimizes both the camera pose and the Gaussian parameters. GS-SLAM proposes a dense visual SLAM system that allocates Gaussians only in regions with high texture, reducing memory consumption while maintaining tracking accuracy [177]. Splat-SLAM performs global bundle adjustment with loop closure detection, optimizing all camera poses and the entire Gaussian cloud simultaneously in a scalable manner [178]. RTGS [84] achieves real-time performance by applying multi-level redundancy reduction: it removes spatially overlapping Gaussians and temporarily freezes the parameters of stable regions, focusing the computation on newly explored areas. OpenGS-SLAM [179] addresses RGB-only Gaussian Splatting SLAM in unbounded outdoor scenes by using pointmap regression for robust pose estimation and jointly optimizing camera poses with the 3DGS map. S3PO-GS [180] extends 3DGS-SLAM by incorporating a global scale factor estimated from monocular depth, compensating for the scale ambiguity inherent in monocular odometry. Gigaslam [181] tackles large-scale monocular SLAM by using a hierarchical representation: a coarse global map for localization and a fine local map for detailed reconstruction. WildGS-SLAM [182] is designed for dynamic environments, introducing robust motion filtering and scene decomposition to track the camera and reconstruct static backgrounds alongside moving objects.

4. Discussion and Analysis

In this section, we summarize the common benchmarks and evaluation protocols that have become standard in the 3D Gaussian Splatting literature and provide a comprehensive analysis of the performance and efficiency of state-of-the-art 3D Gaussian Splatting methods, followed by an in-depth discussion of promising directions for future research.

4.1. Standard Datasets

This subsection organizes commonly used datasets according to their primary evaluation purpose, separating benchmarks for rendering quality, geometry reconstruction, indoor scene reconstruction, large-scale outdoor scenes, and dynamic scene modeling.

4.1.1. Static Novel View Synthesis

Static novel view synthesis is commonly evaluated on Mip-NeRF 360 [183], Local Light Field Fusion (LLFF) [184], Deep Blending [185], and Tanks and Temples [186]. Mip-NeRF 360 contains challenging unbounded scenes with large depth variation and view-dependent effects. LLFF focuses on forward-facing real-world captures with sparse input views. Deep Blending provides free-viewpoint rendering scenes, while Tanks and Temples is frequently used for both view synthesis and reconstruction-oriented evaluation.

4.1.2. Geometry and Indoor RGB-D Scene Reconstruction

Geometry-oriented evaluation commonly uses DTU [187], Tanks and Temples [186], ETH3D [188], ScanNet [189], ScanNet++ [190], Replica [191] and SUN3D [192]. DTU and ETH3D provide calibrated multi-view images with reference geometry and are widely used for multi-view stereo and surface reconstruction. Tanks and Temples provides real-world indoor and outdoor scenes with laser-scanned ground-truth geometry. ScanNet, ScanNet++, and Replica are more commonly used for indoor RGB-D reconstruction, dense geometry evaluation, and scene-level analysis.

4.1.3. Large-Scale Outdoor Scenes

Large-scale outdoor reconstruction and driving-scene rendering are commonly evaluated on KITTI [193], KITTI-360 [194], Waymo Open Dataset [195], and MatrixCity [137]. KITTI and KITTI-360 are widely used for autonomous driving and urban scene understanding. Waymo provides large-scale multi-sensor street-view sequences with dynamic objects and complex traffic scenes. MatrixCity is a synthetic city-scale dataset designed for large-scale neural rendering and reconstruction.

4.1.4. Dynamic Scene Modeling

For dynamic scene modeling and 4D novel view synthesis, the Neural 3D Video dataset [196] is a representative benchmark. It contains multi-view videos of dynamic real-world scenes and is commonly used to evaluate temporal deformation, time-dependent rendering quality, and the efficiency of dynamic 3DGS.

4.2. Evaluation Metrics

The evaluation protocol should match the target task. Rendering metrics measure image quality, geometry metrics assess reconstructed structure, and efficiency metrics quantify the computational and storage cost of the rasterization pipeline.

4.2.1. Rendering Metrics

The standard rendering metrics are PSNR, SSIM [197,198], and LPIPS [199]. PSNR measures pixel-level reconstruction fidelity, SSIM evaluates structural similarity, and LPIPS measures perceptual similarity using deep visual features. Their combination has become the de facto protocol for evaluating novel view synthesis quality in 3DGS research.

4.2.2. Geometry Metrics

For geometry reconstruction, commonly used metrics include Chamfer Distance [200], accuracy, completeness, precision, recall and F-score. Chamfer Distance measures the bidirectional discrepancy between reconstructed and reference point sets or surfaces. Accuracy measures how close the reconstruction is to the reference geometry, while completeness measures how well the reference geometry is covered. Precision, recall, and F-score are widely used in reconstruction benchmarks [186,188].

4.3. Performance Comparison

To facilitate a thorough comparison across different design choices, we evaluate representative methods on three widely adopted benchmarks: Mip-NeRF 360 [183], Tanks & Temples [186], and Deep Blending [185]. The quantitative results are summarized in Table 2. All metrics are computed following the standard evaluation protocol established by 3DGS [8], reporting PSNR, SSIM [197], and LPIPS [199]. For all methods, we conducted experiments using our unified 3D Gaussian Splatting rasterization implementation for fair comparison.

4.3.1. Kernel Representation and Primitive Design

From the upper section of Table 2, it is clear that methods that refine the geometric primitive usually outperform the standard anisotropic 3D Gaussian. Among them, 3D Student Splatting and 3D Skew Gaussian Splatting achieve the highest reconstruction quality across all datasets. These improvements stem from their more flexible kernel shapes: the heavy-tailed Student’s t-distribution provides robustness to outliers, while the skew-normal distribution captures asymmetric radiance distributions that standard Gaussians cannot represent.

Flattening-based methods such as 2DGS and 3D-HGS deliver notable improvements on structured indoor scenes but show a smaller margin on unbounded outdoor environments like Mip-NeRF 360, indicating that they are more suitable for accurate surface extraction tasks. GES, despite its computationally efficient exponential kernel, trades a modest amount of quality for faster rendering, performing on par with 3DGS on Deep Blending but lagging behind other kernel designs on the other two datasets.

4.3.2. Efficiency-Oriented and Compression Methods

The middle section of Table 2 reports methods that prioritize rendering speed and training efficiency. These approaches generally accept a controlled loss in visual quality in exchange for substantial resource savings. Mini-Splatting and FastGS stand out as the most balanced compromises: Mini-Splatting constrains the total Gaussian count through importance-based pruning while preserving competitive PSNR, and FastGS reaches high reconstruction quality despite completing training in under a hundred seconds. In contrast, LightGaussian and Taming-3DGS exhibit larger quality drops, illustrating the difficulty of maintaining fidelity under aggressive compression and limited training budgets.

MMGS attains an order-of-magnitude compression via optimal transport aggregation, retaining accuracy comparable to uncompressed baselines on Tanks & Temples. Speedy-Splat and DashGaussian both deliver real-time frame rates with moderate quality, confirming that jointly exploiting pixel sparsity and primitive sparsity is a viable strategy for interactive applications.

4.3.3. Large-Scale and Anti-Aliasing Methods

The lower section of Table 2 compares methods tailored for large-scale reconstruction or anti-aliased rendering. Wavelet-GS achieves the highest PSNR on Deep Blending, benefiting from frequency-domain compression that preserves high-frequency details while reducing storage. Scaffold-GS and Octree-GS also perform strongly, particularly on Tanks & Temples, confirming that hierarchical and voxelization-based representations effectively handle large, complex scenes.

3DGS-MCMC reaches remarkable reconstruction quality on Mip-NeRF 360, demonstrating that principled, sampling-based densification can match or even surpass hand-crafted heuristics. Mip-Splatting excels in perceptual quality with lower LPIPS scores, confirming that anti-aliasing is crucial for view synthesis. GOF and PGSR show moderate quality but provide accurate surface extraction, a feature essential for downstream tasks such as meshing and collision detection.

4.3.4. Cross-Dataset Observations

Several patterns emerge across the three benchmarks. Deep Blending typically yields the highest PSNR values because of its forward-facing, densely captured scenes. Tanks & Temples poses the greatest challenge: its large-scale outdoor environments with varying illumination and heavy occlusions stress the geometric modeling capacity of each method. Interestingly, methods with alternative kernel designs show the largest relative improvements on Tanks & Temples, suggesting that better geometric primitives are especially beneficial for complex real-world scenes.

4.4. Efficiency Comparison

Low run-time, reduced memory consumption, and compact storage are critical for deployment in resource-constrained settings such as robotics, augmented/virtual reality, and mobile devices [121]. We analyze efficiency along three metrics: primitive counts, training speed and storage reduction.

4.4.1. Gaussian Count and Training Efficiency

We visualize the efficiency trade-offs through a series of plots. Figure 10 reports the Gaussian count compression ratio together with the PSNR improvement over the original 3DGS, highlighting each method’s ability to reduce model size while maintaining or even improving rendering quality. In particular, FastGS achieves a substantial reduction in Gaussian count with almost no quality degradation, while Mini-Splatting strikes an excellent balance between pruning aggressiveness and fidelity. MMGS [89] achieves a 10× reduction via optimal transport aggregation, with a modest quality trade-off. LightGaussian [83] reduces storage but with a larger PSNR penalty, illustrating the challenge of extreme compression.

Figure 11 and Figure 12 use logarithmic axes to better separate closely clustered methods and reveal subtle trade-offs. Figure 11 plots training time against PSNR. The most efficient methods achieve $10\times$ training speed while remaining competitive in quality. FastGS confirms that its sparse rasterization significantly accelerates the optimization loop without incurring a substantial PSNR penalty. Figure 12 compares the final Gaussian count with PSNR, again on logarithmic axes. Most efficiency-oriented methods reduce the number of primitives by 40%–70% relative to 3DGS, yet the quality loss varies significantly. MMGS and FastGS maintain quality close to the original, while LightGaussian accepts a larger drop in exchange for compact models. This indicates that a moderate reduction in primitive count is often achievable with minimal quality impact, but extreme compression still presents an open challenge.

4.4.2. Storage Compression

Figure 13 presents the storage compression ratios achieved by several representative methods across multiple large-scale datasets, including Mip-NeRF 360, Waymo, MatrixCity, and Tanks & Temples. The original 3DGS requires approximately 700–800 MB to store a typical full-resolution scene, which is prohibitive for many applications. HAC++ [162] and PyramidGS [204] achieve impressive storage compression, reducing the model size to a 15–20× reduction while maintaining a competitive PSNR. This is accomplished through a combination of hash-based parameter encoding, structured pruning, and quantization-aware training. Several methods, including Octree-GS [142], Scaffold-GS [141], and ContextGS, reduce storage indirectly by structuring the representation hierarchically or leveraging anchor-based sparsity. Their storage requirements are typically 3–5× smaller than 3DGS while maintaining competitive quality.

4.4.3. Summary of Efficiency Trends

Several key observations can be drawn from this analysis. First, the frontier of efficient 3DGS is advancing rapidly: multiple methods now achieve substantial reductions in training time, primitive count, and storage size without catastrophic quality loss. Second, there is a consistent trade-off between compression ratio and visual quality, but different techniques are pushing the frontier outward. Third, the most successful efficiency methods combine algorithmic innovations like sparse rasterization with implementation-level optimizations such as quantization and gradient clipping. This co-design philosophy is likely to drive further progress toward practical, real-time 3D Gaussian Splatting in everyday devices.

4.5. Future Research Directions

Building upon our rasterization-centric analysis, we identify several critical trajectories for future research, shifting the focus from application-level adaptations to rasterization pipeline innovations.

4.5.1. Sort-Free and Order-Robust Visibility Modeling

The per-tile radix sorting of millions of primitives constitutes a severe latency bottleneck in the current 3DGS pipeline. Furthermore, strict depth ordering frequently induces "popping" artifacts and struggles with complex semi-transparent surfaces. A paramount direction for future rasterizers is the development of sort-free or order-robust visibility modeling. Future frameworks must bypass brute-force sorting, formulating probabilistic or approximate blending mechanisms that maintain geometric fidelity while ensuring strict temporal consistency.

4.5.2. Differentiable Rasterization for Dynamic Visibility

Transitioning explicit representations to 4D dynamics and online SLAM environments introduces profound temporal visibility challenges. Currently, the continuous insertion and deformation during online updates disrupts tile-based memory coherency and necessitates costly global re-sorting. Future rasterization algorithms must natively support dynamic visibility, enabling asynchronous updates to the Gaussian representation. Furthermore, under unposed or sparse-view conditions, the pipeline must robustly propagate gradients to dynamic pose injections and handle depth uncertainty without catastrophic memory fragmentation.

4.5.3. Hardware-Aware Gaussian Rendering Primitives

While algorithmic compression is vital, the theoretical lower bound of rendering latency is strictly governed by hardware utilization. GEMM-GS [118] highlights a massive untapped potential: repurposing Tensor Cores, which is originally designed for neural network inference, for explicit rendering tasks. Future advancements will heavily rely on hardware algorithm co-design, moving toward optimized memory caching and lightweight primitive parameterizations specifically tailored for energy-constrained mobile GPUs [121] and specialized rendering accelerators [116].

4.5.4. Unified Multi-Output Splatting in a Single Pass

ReconFusion [205] and Difix3D+ [206] extended generative inpainting to the 3DGS, is a fertile research direction. Current 3DGS pipelines prioritize RGB novel view synthesis. While recent works embed semantic features [102,103] or physical material properties [94], these are typically achieved through computationally expensive, post-hoc optimization passes or duplicated rendering loops. A critical architectural evolution is the realization of a unified rasterizer. Such a pipeline would simultaneously accumulate color, depth, high-frequency surface normals, semantic logits, and BRDF properties within a single forward pass, dynamically sharing intermediate visibility states to avoid linear scaling of computational costs.

4.5.5. Standardized Efficiency and Systems-Level Benchmarks

As the field of explicit volumetric rendering matures, traditional evaluation metrics limited to perceptual quality are insufficient to capture the nuanced trade-offs of rasterization algorithms. Future research necessitates the establishment of standardized, systems-level benchmarks. To comprehensively evaluate a proposed rasterization intervention, protocols must rigorously quantify peak VRAM footprint during forward/backward passes, per-tile sorting latency, storage compression efficiency, and mobile-device energy cost. This paradigm shift will ensure that algorithmic innovations translate into tangible engineering deployments.

5. Conclusion

In this survey, we presented a rasterization-centric review of 3D Gaussian Splatting, organizing recent advances according to their intervention points in the rendering data flow. By deconstructing the 3DGS pipeline into representation design, tile-based scheduling, visibility sorting, alpha blending, and hardware-aware optimization, we illuminated the fundamental principles of how rasterization algorithms shape the efficiency and scalability of modern 3DGS systems. Furthermore, through a rigorous analysis of computational bottlenecks, we demonstrated how targeted innovations in rasterization algorithms serve as the primary catalyst for performance gains across diverse tasks. Finally, we further revealed that future progress will depend not only on more expressive primitives, but also on tighter co-design among algorithms, memory layouts, and hardware execution models.