BrushSizzleCompressedFrame.mp4
Massive thanks to @GradeEterna for the beautiful scenes
Brush is a 3D reconstruction engine using Gaussian splatting. It works on a wide range of systems: macOS/windows/linux, AMD/Nvidia/Intel cards, Android, and in a browser. To achieve this, it uses WebGPU compatible tech and the Burn machine learning framework.
Machine learning for real time rendering has tons of potential, but most ML tools don't work well with it: Rendering requires realtime interactivity, usually involve dynamic shapes & computations, don't run on most platforms, and it can be cumbersome to ship apps with large CUDA deps. Brush on the other hand produces simple dependency free binaries, runs on nearly all devices, without any setup.
Try the web demo 
Brush takes in COLMAP data or datasets in the Nerfstudio format. Training is fully supported natively, on mobile, and in a browser. While training you can interact with the scene and see the training dynamics live, and compare the current rendering to input views as the training progresses.
It also supports masking images:
- Images with transparency. This will force the final splat to match the transparency of the input.
- A folder of images called 'masks'. This ignores parts of the image that are masked out.
Black pixels in the mask are ignored, white pixels are kept. Pass
--invert-masksif your masks are the other way around.
For captures with varying exposure, white balance, or lens vignetting between images, Brush can learn per-view photometric corrections during training so the variation isn't baked into the splats. The corrections only apply while training — exported splats keep canonical colors and render unmodified.
--bilateral-gridlearns a per-view affine color grid using gsplat's Apache-2.0 bilateral-grid semantics.--ppispenables the full NVIDIA PPISP model: per-frame exposure and color plus per-camera vignetting and tone curves.--ppisp-gridenables the Spirulae-Splat-style hybrid: per-camera vignetting followed by a per-view bilateral grid of exposure, color, and tone-curve parameters. Use--ppisp-grid-expose-only,--ppisp-grid-crf=false, or--ppisp-crf-per-camerato select payload variants.
Choose one appearance model per training run. The three flags are mutually exclusive because stacking the models introduces redundant, poorly identified corrections.
Tunables: --bilagrid-dims x,y,guidance, --bilagrid-tv-weight, --bilagrid-mean-reg, --bilagrid-lr, --bilagrid-betas b1,b2, --bilagrid-grad-subsample, --ppisp-lr, --ppisp-reg-scale.
By default evaluation compares the raw, uncorrected render against ground truth — on appearance-varying captures that mostly measures the offset between the splats and the average appearance. Pass --train-on-eval to keep eval views in the training set; eval then applies each view's learned correction, which is the more meaningful comparison for these models.
Appearance parameters are training-only and are not stored in PLY checkpoints or used for novel-view rendering. Resuming at a non-zero iteration with appearance compensation is rejected to avoid silently resetting them.
Brush also works well as a splat viewer, including on the web. It can load .ply & .compressed.ply files. You can stream in data from a URL (for a web app, simply append ?url=).
Brush also can load .zip of splat files to display them as an animation, or a special ply that includes delta frames (see cat-4D and Cap4D!).
Brush can be used as a CLI. Run brush --help to get an overview. Every CLI command can work with --with-viewer which also opens the UI, for easy debugging.
For lightweight machine-readable training telemetry, set BRUSH_METRICS_LOG
to a JSONL output path. Brush replaces that file for each run and writes a run
header plus the first, final, and every 50th training step. Override the cadence
with a positive BRUSH_METRICS_EVERY value.
rerun_dash_compressed.mp4
While training, additional data can be visualized with the excellent rerun. To install rerun on your machine, please follow their instructions. Open the ./brush_blueprint.rbl in the viewer for best results.
First install rust 1.88+. You can run tests with cargo test --all. Brush uses the wonderful rerun for additional visualizations while training, run cargo install rerun-cli if you want to use it.
Use cargo run --release from the workspace root to make an optimized build. Use cargo run to run a debug build.
On Apple Silicon macOS, the desktop brush app compiles native Metal Shading
Language code generation by default. Build and run it normally:
cargo build --release
./target/release/brushThe native-MSL optimization preset is also enabled by default for that build, so no launch environment is required. To build the portable WGSL desktop app for CI, a virtualized Mac, or unsupported hardware, disable default features:
cargo build --release --no-default-features
./target/release/brushbrush-cli and brush-c retain explicit host-controlled compiler selection;
build them with --features native-msl when appropriate. On non-Metal backends
the feature continues to use WGSL. The compiler choice applies to the whole
binary, so compare WGSL and MSL with separate builds.
The runtime preset requests all six retained native-MSL training optimizations. An explicit environment value still overrides the Apple Silicon default. For example, this keeps native MSL code generation but disables the preset:
BRUSH_NATIVE_MSL_PRESET=0 ./target/release/brushThe preset is equivalent to setting these individual options to 1:
BRUSH_NATIVE_MSL_UNCHECKED_RASTER_BWDBRUSH_NATIVE_MSL_FUSED_SH_ADAMBRUSH_NATIVE_MSL_COALESCED_SH_GRADBRUSH_NATIVE_MSL_SAVED_LOSS_PARTIALSBRUSH_NATIVE_MSL_SPARSE_SH_ADAMBRUSH_NATIVE_MSL_FINE_RASTER_TILES
Each option remains subject to its compile-time, tensor-shape, and device capability checks; unsupported cases retain the existing implementation. An explicit per-option value overrides the preset, which is useful for isolation or memory-constrained runs:
BRUSH_NATIVE_MSL_PRESET=1 \
BRUSH_NATIVE_MSL_SAVED_LOSS_PARTIALS=0 \
./target/release/brushOnly 1 and case-insensitive true enable a switch. 0, case-insensitive
false, or an unrecognized explicit value disable it. The preset defaults on
only for Apple Silicon macOS native-MSL builds; elsewhere it remains off. All
options remain subject to their required compile-time and platform gates.
The preset selects the 16x8 training rasterizer on Apple Silicon native-MSL builds. It replaces the 16x16 tile geometry for the forward, map, raster, and backward training passes as one unit; product rendering entry points continue to use 16x16 tiles. The standalone option remains available without the rest of the preset:
BRUSH_NATIVE_MSL_FINE_RASTER_TILES=1 cargo run --releaseFine tiles alone retain bounds checks in raster backward; the full preset also
requests BRUSH_NATIVE_MSL_UNCHECKED_RASTER_BWD and uses the same
host-validated unchecked launch contract as 16x16. To isolate the old geometry
or work around a device-specific issue, explicitly disable fine tiles while
retaining the rest of the preset:
BRUSH_NATIVE_MSL_PRESET=1 \
BRUSH_NATIVE_MSL_FINE_RASTER_TILES=0 \
./target/release/brushThe 16x8 performance, memory, gradient, six-run 15k quality, and 30k stability gates are recorded in the fine-tile results.
Native-MSL builds also expose an experimental, off-by-default raster-backward path without generated buffer bounds checks. It relies on the renderer's tile/range invariants and requires native float atomics (otherwise it falls back to the checked path), so use it for controlled benchmarking and soaks rather than production builds:
BRUSH_NATIVE_MSL_UNCHECKED_RASTER_BWD=1 cargo run --releaseAn experimental fused update for the spherical-harmonic Adam state is also available on Apple Silicon native-MSL builds. It preserves the existing per-coefficient learning-rate scaling and reduced second-moment state, and falls back to the generic optimizer for unsupported tensor shapes or devices:
BRUSH_NATIVE_MSL_FUSED_SH_ADAM=1 cargo run --releaseAn experimental Apple Silicon native-MSL path can also coalesce dense spherical-harmonic gradient materialization. This path preserves exact zero rows for splats that do not contribute to the sampled view, so optimizer momentum decay and the dense gradient contract remain unchanged. It falls back to the existing path when the required 32-lane SIMD-group support is unavailable:
BRUSH_NATIVE_MSL_COALESCED_SH_GRAD=1 cargo run --releaseThe steady-state Apple Silicon path can instead keep spherical-harmonic gradients sparse and fuse their reconstruction directly into the reduced Adam update. It falls back to the dense gradient and optimizer paths when the model, optimizer state, or device is incompatible. During compatible steady-state steps this supersedes the coalesced dense-gradient and fused dense-Adam paths; the first step remains dense to initialize Adam state (and may use coalesced gradient materialization), while both dense options remain available on later sparse fallback steps:
BRUSH_NATIVE_MSL_SPARSE_SH_ADAM=1 cargo run --releaseTracked SSIM training can optionally save the three f32 SSIM partials from
forward for reuse by backward. This removes the first image-load and blur pair
from loss backward without changing its formulas, but adds a [9, H, W] tape
tensor of 36 bytes per pixel: about 71.2 MiB at 1920x1080 and 284.8 MiB at
3840x2160. Eval, untracked, L1-only, non-Apple-Silicon, and default builds
continue to use the recompute path. The
1440x1920 egg replay uses about 94.9 MiB for this tape, so disable this option
explicitly under the preset on memory-constrained systems. Its standalone
opt-in remains:
BRUSH_NATIVE_MSL_SAVED_LOSS_PARTIALS=1 cargo run --releaseBrush can be compiled to WASM. Run npm run dev to start the demo website using Next.js, see the web directory in app/brush-app/web.
Brush uses wasm-pack to build the WASM bundle. You can also use it without a bundler, see wasm-pack's documentation.
WebGPU is still an upcoming standard, and as such, only Chrome 134+ on Windows and macOS is currently supported.
As a one time setup, make sure you have the Android SDK & NDK installed.
- Check if ANDROID_NDK_HOME and ANDROID_HOME are set
- Add the Android target to rust
rustup target add aarch64-linux-android - Install cargo-ndk to manage building a lib
cargo install cargo-ndk
Each time you change the rust code, run
cargo ndk -t arm64-v8a -o crates/brush-app/app/src/main/jniLibs/ build- Nb: Nb, for best performance, build in release mode. This is separate from the Android Studio app build configuration.
cargo ndk -t arm64-v8a -o crates/brush-app/app/src/main/jniLibs/ build --release
You can now either run the project from Android Studio (Android Studio does NOT build the rust code), or run it from the command line:
./gradlew build
./gradlew installDebug
adb shell am start -n com.splats.app/.MainActivity
You can also open this folder as a project in Android Studio and run things from there. Nb: Running in Android Studio does not rebuild the rust code automatically.
Rendering and training are generally faster than gsplat. You can run benchmarks of some of the kernels using cargo bench.
To benchmark native MSL code generation on macOS, run cargo bench -p brush-bench-test --features native-msl.
For a steady-state replay using an exported checkpoint and real dataset views, use the standalone benchmark binary. Setup, image decoding, pipeline compilation, and optimizer initialization happen before timing:
BRUSH_NATIVE_MSL_PRESET=1 \
cargo run --release -p brush-bench-test --bin brush-checkpoint-replay --features native-msl -- \
--dataset /path/to/dataset \
--ply /path/to/checkpoint.ply \
--eval-split-every 20The replay restores model parameters but starts fresh optimizer state. It is intended to reproduce geometry-, visibility-, and resolution-dependent GPU work, not to resume training numerically from the checkpoint.
The replay reports the preset and each resolved per-option request. These fields show configuration intent; device and workload gates can still select a fallback implementation.
Pass --skip-refine-weight to benchmark the late phase after high-gradient
densification stops. Production training selects that path automatically at
--growth-stop-iter; visibility and screen-radius refinement stats remain enabled.
To inspect raster workload shape without changing any GPU kernel, build the
replay with the diagnostics-only raster-census feature. The first untimed
warmup cycle reads back the existing projected splats, intersection list, and
tile offsets. It reports exact tile occupancy and backward atomic fan-in plus a
deterministic CPU replay of the requested number of tiles:
cargo run --release -p brush-bench-test \
--bin brush-checkpoint-replay --features native-msl,raster-census -- \
--dataset /path/to/dataset \
--ply /path/to/checkpoint.ply \
--max-resolution 1920 \
--views 4 \
--warmup-steps 4 \
--steps-per-sample 4 \
--samples 1 \
--raster-census-tiles 256Each view emits one BRUSH_RASTER_CENSUS JSON record. Census readbacks are
synchronous and deliberately excluded from normal builds; do not use a census
run for timing. Capture CubeCL timestamp profiles in a separate ordinary replay
without the raster-census feature. The
egg raster workload census
records the measurements that selected the first fine-tile candidate.
For post-hoc quality evaluation, render the held-out dataset views from an exported PLY with the standalone evaluator. Alpha interpretation is required so comparisons cannot silently use different masking behavior:
cargo run --release -p brush-bench-test --bin brush-eval-checkpoint --features native-msl -- \
--dataset /path/to/dataset \
--ply /path/to/checkpoint.ply \
--eval-split-every 20 \
--alpha-mode masked \
--save-dir /path/to/rendersThe evaluator emits one BRUSH_EVAL_VIEW JSON record per held-out view and one
aggregate BRUSH_EVAL_RESULT record. Existing psnr and ssim fields remain
full-frame metrics. When --alpha-mode masked is used with alpha-bearing
images, the records also include mask-normalized metrics and coverage; empty
masks are reported and excluded from masked aggregates. Masked SSIM weights
window centres by alpha, so RGB outside the mask can still influence windows
that cross the boundary. See the
egg 15k upstream-versus-macOS-preset bake-off
for the frozen performance and quality baseline used by raster redesign work.
gSplat, for their reference version of the kernels
Peter Hedman, George Kopanas & Bernhard Kerbl, for the many discussions & pointers.
The Burn team, for help & improvements to Burn along the way
Raph Levien, for the original version of the GPU radix sort.
GradeEterna, for feedback and their scenes.
This is not an official Google product. This repository is a forked public version of the google-research repository