μ
Mean position
The center of the Gaussian in 3D space. It tells the renderer where the blob lives.
Definition
What is Gaussian Splatting?
Gaussian Splatting is a 3D scene representation and rendering method. It learns many small, transparent, colored Gaussian blobs from calibrated photos, then projects those blobs into the camera view and blends them to create a new image.
The core idea
A Gaussian is a smooth probability-shaped blob. In 3D Gaussian Splatting, each blob acts like a tiny soft piece of visible matter. Each one stores where it is, how stretched it is, how transparent it is, and what color it emits from different viewing directions.
The scene is not a mesh. It is also not hidden inside a neural network. It is an explicit set of optimized primitives that can be projected quickly onto the screen.
G(x) = exp(-1/2 · (x - μ)T Σ-1 (x - μ))
Think of it as painting with transparent 3D airbrush strokes.Many overlapping ellipsoids approximate surfaces, edges, color, and view-dependent appearance.
Σ
Covariance
The size, orientation, and stretch of the ellipsoid. This is why splats can become thin, wide, or elongated.
α
Opacity
How much the Gaussian contributes to the final pixel. Transparent blending is essential to the method.
RGB
Color model
Color is often stored with spherical harmonics, allowing the same splat to look different from different directions.
Pipeline
From photos to a splatted scene
The method begins like photogrammetry: capture several images, estimate camera poses, and create an initial sparse point cloud. Then the Gaussian parameters are optimized so rendered views match the original photos.
1
Capture images
Take overlapping photos or frames of a mostly static scene.
2
Estimate cameras
Structure-from-motion recovers camera poses and sparse points.
3
Create Gaussians
Initialize one or more 3D Gaussians around the sparse points.
4
Optimize
Adjust position, size, rotation, opacity, and color to match photos.
5
Control density
Split, clone, grow, or prune splats where detail is missing or unnecessary.
6
Render views
Project splats into the screen and alpha-blend them in visibility order.
Interactive demo
Build an image from splats
This 2D toy demo shows the rendering intuition. Each circle or ellipse is a soft Gaussian contribution. The final image is built by accumulating many semi-transparent splats rather than drawing hard polygons.
Gaussian splats · 120
In real 3D Gaussian Splatting, these are projected ellipses from 3D ellipsoids. The renderer sorts or tiles visible splats and blends them using alpha compositing.
Rendering
How splatting produces a pixel
Rendering means converting the 3D Gaussian cloud into a 2D image. The important trick is that each 3D ellipsoid becomes a 2D elliptical footprint in the camera view. The renderer evaluates and blends the splats that touch each pixel.
Camera
1
Project
Transform each 3D Gaussian into camera space and approximate its footprint as a 2D ellipse.
2
Cull and tile
Discard invisible splats and group remaining splats by screen tiles for fast GPU processing.
3
Sort for visibility
Use depth ordering so closer splats correctly affect what is visible in front of farther splats.
4
Alpha blend
Combine color and transparency until the pixel becomes the final rendered image value.
5
Differentiate
During training, compare the rendered image with real photos and update Gaussian parameters.
Density control
Why “splitting” matters
Although the technique is called Gaussian Splatting, splitting is an important optimization idea: when a Gaussian is too large or cannot explain image detail, it can be split into smaller Gaussians. Weak or unnecessary splats can also be pruned.
✂
Split
A large or high-error Gaussian can become several smaller ones, improving detail around edges, corners, foliage, or thin structures.
⧉
Clone
Useful Gaussians may be duplicated near areas where the optimizer needs more local capacity.
⌫
Prune
Nearly invisible, unstable, or redundant Gaussians can be removed to reduce memory and improve performance.
Comparison
How it differs from other 3D methods
Gaussian Splatting sits between classic graphics and neural rendering. It has the explicit, rasterization-friendly nature of graphics primitives, but it learns appearance from photos like neural radiance field methods.
| Method | Representation | Strengths | Weaknesses | Best fit |
|---|---|---|---|---|
| Mesh rendering | Triangles, materials, textures | Editable, standard in games/CAD, physically meaningful surfaces | Hard to reconstruct perfect real scenes automatically; transparency and complex appearance can be difficult | Games, engineering, animation, product visualization |
| Photogrammetry | Dense geometry plus texture maps | Produces measurable geometry and traditional assets | Can struggle with reflections, textureless areas, holes, and heavy cleanup | Surveying, heritage, asset capture |
| NeRF | Implicit neural radiance field | High visual quality and elegant continuous representation | Often slower to render because many samples and network evaluations are required | Research, view synthesis, compact learned scenes |
| 3D Gaussian Splatting | Explicit cloud of optimized transparent ellipsoids | Excellent visual quality, fast rendering, direct rasterization-like pipeline | Memory can be high; editing and simulation are less mature than meshes | Real-time captured scenes, VR previews, digital twins, immersive media |
Applications
Where Gaussian Splatting is useful
Its main strength is fast, high-quality novel-view synthesis: moving a virtual camera through a captured real place or object while preserving photographic appearance.
🏛
Cultural heritage
Capture monuments, rooms, artifacts, and excavation sites as immersive navigable scenes.
photoreal capturearchives🏠
Real estate and architecture
Create walk-throughs from photo or video capture without manually modeling every surface.
virtual toursAEC🎬
Film and virtual production
Rapidly turn real environments into background assets for previs, VFX, and scene blocking.
VFXprevisualization🥽
VR and AR
Real-time rendering makes captured spaces more practical for immersive experiences.
immersivereal time🤖
Robotics and simulation
Captured radiance fields can help with visual scene understanding and synthetic viewpoint generation.
mappingtraining data🛒
Product visualization
Objects can be captured with rich appearance and inspected interactively from many angles.
e-commerce3D assetsLimitations
What it does not solve perfectly
Gaussian Splatting is powerful, but not magic. It works best when input views are well-covered, camera poses are accurate, and the scene is mostly static and consistently lit.
!
Dynamic scenes
Moving people, changing lights, water, vegetation, and traffic can create inconsistent training signals.
!
Reflective and transparent materials
Mirrors, glass, glossy metals, and refractions can be difficult because they change strongly with viewpoint.
!
Weak image coverage
Unseen backsides, narrow gaps, or badly covered angles may produce holes, floaters, or blurry regions.
!
Large memory footprint
High-quality scenes may require many Gaussians, which can be heavier than a compact neural representation.
!
Editing is not mesh editing
Moving walls, changing object topology, or adding physical simulation is harder than with triangle meshes.
!
Geometry is approximate
The result may look photorealistic but still lack clean, watertight, measurable surfaces.
FAQ
Important questions
These answers summarize the most important practical concepts behind the technique.
Is Gaussian Splatting the same as NeRF?
No. Both are used for novel-view synthesis, but NeRF commonly stores a scene inside an implicit neural network, while 3D Gaussian Splatting stores an explicit set of optimized Gaussian primitives that can be rasterized quickly.
Why are the Gaussians anisotropic?
An anisotropic Gaussian can stretch differently in different directions. That lets one primitive approximate a flat surface patch, an elongated edge, or a thin visual feature better than a simple sphere.
What is a splat?
A splat is the screen-space footprint of a primitive. In this method, a 3D Gaussian projects to a soft 2D ellipse, and that ellipse contributes color and opacity to pixels.
Does it create a real mesh?
Not directly. The output is a cloud of Gaussian ellipsoids. Mesh extraction is possible with additional processing, but the native representation is not a triangle mesh.
Why can it render so fast?
The explicit Gaussian primitives can be processed with a rasterization-like GPU pipeline. The renderer avoids sampling a dense volume everywhere and focuses on visible splats.
Glossary
Key terms
A compact vocabulary for reading papers, code, and tutorials about Gaussian Splatting.
Radiance fieldA function describing color and density/opacity of a scene from different positions and directions.
Novel-view synthesisRendering a scene from a new camera position that was not part of the original input images.
Spherical harmonicsA compact basis often used to encode view-dependent color, such as subtle highlights.
Alpha compositingThe process of blending semi-transparent colors from front to back or back to front.
Structure from MotionA computer vision process that estimates camera positions and a sparse point cloud from images.
Covariance matrixThe mathematical object controlling the Gaussian ellipsoid’s shape, scale, and orientation.
Tile rasterizerA GPU-friendly renderer that groups work by screen regions for speed and parallelism.
Density controlAdding, splitting, cloning, or pruning Gaussians during optimization to balance quality and efficiency.
FloatersUnwanted cloudy artifacts that appear where the reconstruction is uncertain or poorly constrained.