Diffusion on Meshes and DMTets

The Representation ProblemConnectivity Is Discrete

Adding Gaussian noise to an image is straightforward because every sample keeps the same address. A raw mesh can change vertex count, face count, and adjacency. Noise on vertex coordinates alone assumes corresponding topology across the dataset—a strong restriction.

DMTetA Fixed Tetrahedral Scaffold

A tetrahedral grid divides volume into small tetrahedra. Store a signed distance value at grid vertices and optionally deform their positions. Where the sign changes across tetrahedron edges, marching tetrahedra extracts surface triangles. The grid provides a stable tensor structure while the zero level set provides flexible topology.

MeshDiffusionDiffuse the Direct Parameterization

MeshDiffusion trains a diffusion model over deformable marching-tetrahedra parameters. The project demonstrates unconditional generation, completion from a partial single view, interpolation using DDIM sampling, and subsequent text-conditioned texture generation.

Training IntuitionNoise, Predict, Reverse

# High-level pseudocode—not the official implementation.
state_0 = encode_mesh_as_dmtet(mesh)
t = random_timestep(batch_size)
epsilon = torch.randn_like(state_0)
state_t = alpha[t].sqrt() * state_0 + (1 - alpha[t]).sqrt() * epsilon

predicted_noise = denoiser(state_t, t, condition)
loss = (predicted_noise - epsilon).square().mean()

# Sampling begins from noise and repeatedly applies the learned reverse step.
state = torch.randn_like(template_state)
for t in reversed(schedule):
    state = reverse_diffusion_step(denoiser, state, t, condition)
mesh = marching_tetrahedra(state)

What the Grid BuysTopology Without Autoregressive Faces

The model does not need to emit a syntactically valid face sequence one triangle at a time. It predicts a field-like representation and lets a deterministic extractor build triangles. This supports topology changes and parallel denoising, but surface resolution remains tied to the tetrahedral scaffold and extraction.

What It Does Not BuyArtist Topology and Printability

An extracted isosurface can be dense and mechanically indifferent. Edge loops, UV seams, watertight components, minimum walls, flat contact regions, and manufacturing tolerances still need explicit processing. “Direct mesh generation” in a paper can mean the final output is a mesh; it does not necessarily mean human-authored polygon flow.

ConditioningCompletion Is a Posterior, Not Ground Truth

Single-view conditioning narrows the distribution but does not reveal the back of an object. Preserve uncertainty by generating alternatives and comparing silhouette, multi-view consistency, symmetry, and process constraints. A deterministic production system can route only accepted candidates into expensive texturing and repair.

Hybrid TacticFields for Shape, Tokens for Structure

A field or sparse-voxel model is strong at broad topology and detailed surfaces. A mesh tokenizer is strong at compact polygonal structure. A useful hybrid generates or reconstructs shape first, samples a clean conditioning point cloud, then asks an autoregressive mesh model for artist-like topology and validates it against the source surface.