Voxelforge

A real-time dynamic voxel world with two interchangeable render backends — primary Gaussian-surfel rasterization and a chunked sparse-voxel octree (SVO) sphere tracer kept as the pixel reference — where a local AI can add, remove and shape voxel objects at runtime through chat or MCP tool calls, without ever touching source code.

Every piece of geometry — rolling terrain, the lake, the hamlet's hall, tower, pier, boat and props, anything the AI builds — lives in plain .vxw voxel-record files. The renderer bakes those records into GPU octrees and surfels on the fly and hot-reloads them while running.

Gaussian surfels (--mode splat, default)	SVO raymarcher (--mode svo, reference)

Same world, same camera — only the geometry backend differs; F toggles it live. See the full renderer comparison for side-by-side views, pixel-level zooms and measured timings.

Features

• Gaussian-surfel rendering (primary backend, --mode splat, default): the CPU bake (src/voxel/surfelize.*) extracts one 80 B 2D Gaussian disk per outer surface cell — position, normal, baked sun shadow, bent-normal AO, material — plus deterministic micro-detail children that turn texture texels into real geometry. Instanced quads rasterize through a depth prepass + opaque base + Gaussian band, so surfaces stay watertight while silhouettes are soft and anti-aliased; frustum culling, GPU compaction, a Hi-Z occlusion pyramid and baked terrain LOD rings keep object-heavy and close-up views fast ([/] tunes disk size live). Same shading model as the reference — see the renderer comparison.
• SVO raymarcher (pixel reference, --mode svo): chunked sparse-voxel octree sphere tracing (16³ chunks, 8³ bricks, 2 × uint32 per voxel: RGB + SDF, alpha/refl/rough/material) with exact per-pixel shadows and AO. Used to validate shading and A/B every change; F toggles backends live.
• Records-only geometry (src/voxel/voxel_field.*): terrain comes from per-column landscape records, objects from connected components flood-filled to solid volumes with a signed distance transform. No analytic scene code in the runtime or the shader.
• Physically-flavoured shading: ACES tonemapping, PBR-style specular, soft shadows from the smooth terrain heightfield + a coarse object-only SDF volume (so AI-placed objects cast correct shadows), bent-normal AO, Preetham-style sky, altitude fog, animated water with refraction/foam and underwater volumetrics, wind-sheared grass and foliage micro-geometry.
• TAA in interactive mode.
• Live world editing: layer toggles, MCP appends and chat edits land as .vxw file changes; the app polls mtimes and swaps freshly synthesized SVO buffers in-place (~0.5 s). Chat edits trigger the reload immediately.
• AI tooling: in-app chat window plus a stdio MCP server (vf_mcp) exposing add_box / add_cylinder / add_ellipsoid / add_stamp / list_layers / enable_layer / probe / ground / clear_edits.
• STL/OBJ authoring in the GUI: Dashboard/World Layers → Import STL / OBJ scans assets/models/, voxelizes a selected mesh onto the 10 cm lattice, and writes a named .vxw layer. Fit/scale, source-axis/winding, material, solid/shell, and anchor controls are exposed; replacing a placed layer preserves its manifest pose/orientation.
• Authoring tools: vf_slice prints ASCII cross-sections of the baked field (including AI edits); --probe answers point queries.

Requirements

• Linux, Vulkan 1.3 (dedicated GPU recommended)
• CMake ≥ 3.24, Ninja, a C++20 compiler, Python 3 (for tests)
• glslangValidator
• An OpenAI-compatible endpoint or Ollama for the chat backend (optional)

All libraries (spdlog, GLFW, glm, VMA, doctest, ImGui) are fetched automatically.

Build & first run

cmake -S . -B build -G Ninja
ninja -C build          # binaries + shaders
ninja -C build world    # bake heightmap + .vxw layers + world.json (once)
./build/voxelforge

The app refuses to start without baked assets — ninja -C build world runs the offline baker (tools/heightmap_gen.cpp), which writes:

asset	role
assets/heightmap.png	baker-side terrain source (2048², 5 cm/texel)
assets/landscape.vxw	terrain shell records (per lattice column)
assets/hamlet_*.vxw	runtime-authored object layers (hall, imported cabin, tower, pier, boat, well, market, garden, pines, props) written by vf_mcp or the GUI mesh importer
assets/world.json	layer manifest — order = dedupe priority, first wins a cell
assets/ai_edits.vxw	highest-priority live layer for chat/MCP edits

The app starts in the lakeside hamlet: landscape + the hamlet_* object layers (plus your AI edits) are loaded. The "World layers" panel in-app lists every .vxw file as a checkbox — tick hamlet_tower.vxw, … to load it into the running world; untick to remove it. New files dropped into assets/ appear after a rescan.

Layers store absolute world coordinates, so a checkbox materializes the object exactly where it was baked. To place a copy somewhere else, pick an anchor with Ctrl+LMB and hit the layer's Import button — it stamps the object at the selection (bottom-center) by appending translated records to ai_edits.vxw, leaving the original layer untouched. For triangle assets, use Import STL / OBJ in the same panel; the current hamlet_cabin is reimported from assets/models/Forrest_Hunting_Cabin.stl without changing its authored world.json orientation.

Controls

input	action
WASD / Q E	move down/up
RMB + mouse	look
wheel	scroll the sidebar
Shift / Ctrl	fly 3.5x faster / 4x slower (base speed: Render panel -> View -> Speed)
Ctrl+LMB	pick a voxel → becomes the bottom-center anchor for AI builds
C → Rotate	select the rotation tool
plain LMB in Rotate	activate the exact object under the cursor
click-drag trackball ring	local Y (outer), local X (wide), or local Z (tall); release stages, then press Apply
C → Move	grab the exact object under the cursor
C → Smooth	relax nearby terrain column heights; Size sets the footprint and Strength sets relaxation per stamp
drag in Move	translate along the selected world X/Y/Z handle; release stages, then press Apply
window close button	quit (Esc is reserved and does not exit)

AI editing

In-app chat

The AI Chat window talks to any Ollama/OpenAI-compatible server. Pick an anchor voxel with Ctrl+LMB and ask for "a 3×3 wood crate", "a small rock", "a stamp shaped like an arch"… The model's tool calls are normalized (fuzzy names, inline JSON, escaped args), executed against EditableWorld, and the world reloads instantly.

VF_LLM_URL=http://127.0.0.1:11434 VF_LLM_MODEL=qwen2.5 ./build/voxelforge
VF_LLM_URL=http://localhost:8080/v1 ./build/voxelforge   # llama.cpp / vLLM …

MCP server

Register once (already wired for opencode in .opencode/opencode.json):

{ "command": "./build/vf_mcp", "args": [] }

tool	effect
ground {x,z}	terrain height + suggested anchor cell
probe {x,y,z}	signed distance + material at a point
add_box / add_cylinder / add_ellipsoid / add_stamp	append records to ai_edits.vxw (anchored at "anchor":[x,y,z] or "ground":[x,z])
list_layers / enable_layer	inspect/toggle manifest layers
clear_edits	wipe ai_edits.vxw
import_mesh	convert an STL/OBJ into a named .vxw layer (GUI, CLI, and MCP share the pipeline)

Edits are visible to a running instance within ~0.5 s — no repack, no rebuild.

Command line

--selftest              headless acceptance check (sky probe + coverage)
--smoke N               run N frames headless, report timings
--shot out.ppm          render one frame and exit
--cam X Y Z TX TY TZ    camera placement
--sun ELEV AZIM         sun direction (degrees)
--animtime S            deterministic water/grass animation clock
--probe X Y Z           print field distance/material at a point, exit
--width/--height        offscreen resolution for headless modes
--llm-url/--llm-model   chat backend override

Testing

Tests are organized into focused, opt-in groups. Build once, then run only the group that covers the change; do not run a bare all-tests command.

ninja -C build
ninja -C build test-surfel       # surfelize / surfel geometry
ninja -C build test-live-edit    # ChunkStore and live patches
ninja -C build test-visual       # camera / shader / scene acceptance
ninja -C build test-store        # store foundations and rebuilds
ninja -C build test-world        # layered world / SVO / records
ninja -C build test-smoke        # quick cross-area check; test-fast alias

Additional groups: test-unit, test-effects, test-textures, and test-fog. CTest entries are group-gated; a bare ctest --test-dir build skips them and there is no all-tests target.

Unit tests cover the authoring primitives, stamp semantics, VXW round-trip and corruption handling, camera math, chat-tool normalization, and cross-check the baked VoxelField against the analytic authoring truth. visual_check renders the hero/house/water shots and asserts coverage, silhouette and sky probes.

Architecture map

src/
  voxel/voxel_field.{hpp,cpp}  records-derived geometry oracle:
                               terrain columns, EDT-flood-filled object grids,
                               sparse object SDF hash, coarse shadow volume
  voxel/layered_world.{hpp,cpp} manifest load/poll, priority merge, SVO synthesis
  voxel/editable_world.{hpp,cpp} ai_edits.vxw writer (box/cylinder/ellipsoid/stamp)
  voxel/worldfile.{hpp,cpp}     VXW v1 container (64 B header + CRC32)
  voxel/common.hpp              constants, palette, BAKER-side analytic shapes
  ai/ollama_client.cpp          HTTP chat client + tool-call parsing
  ai/mcp_server.cpp             vf_mcp stdio JSON-RPC server
  app/chat_ui.cpp               ImGui chat window + tool dispatch
  app/main.cpp                  Vulkan app: swapchain, frames, HUD, picking
  render/svo_pass.cpp           SVO compute pipeline (+ push block, descriptors)
  render/taa_pass.cpp           temporal anti-aliasing resolve
  rhi/*                         context, buffers/images, swapchain
tools/heightmap_gen.cpp         offline asset baker (terrain + authored layers)
tools/scene_slice.cpp           ASCII cross-sections of the baked field

Shader truth lives in shaders/svo_raymarch.comp: terrain is sampled from a records-derived rg32f height texture (top Y + material), objects from bricks; shadow rays march the smooth heightfield plus a coarse object-SDF volume. There are no analytic scene constants in GLSL by design — new geometry only ever arrives as data.

Documentation

Full developer documentation lives in docs/:

doc	contents
docs/getting-started.md	build, asset bake, controls, headless modes, troubleshooting
docs/architecture.md	data flow, module tour, synthesis & hot-reload model
docs/world-format.md	VXW v1 binary spec, manifest schema, layer merge rules
docs/rendering.md	GPU contract: handles, brick packing, push block, textures
docs/rendering-comparison.md	surfels vs SVO side by side: screenshots, zooms, timings
docs/ai-editing.md	chat backend, tool normalization, MCP protocol reference
docs/tooling.md	full CLI/env reference, vf_slice, baker, start.sh
docs/testing.md	test gates, suite breakdown, debug workflows
docs/contributing.md	invariants, conventions, gotchas, how-to recipes

Historical design docs (records-only rework plan, roadmap, session log) live in docs/history/. Engineering conventions for coding agents: AGENTS.md.