Displacement mapping for StandardMaterial, composable with parallax

[mvaligursky]

Supersedes #5554, which prototyped vertex displacement by replacing transformVS and ran into the shadow-pass problems described below. This is a clean write-up of the design, aligned with the parallax stack that landed for 2.22 (#9213 occlusion mode, #9225 self shadowing, #9245 configurable zero level), which was shaped with this feature in mind.

Proposed API

Displacement should be a separate property from parallaxMode, not a third value of it, because the two compose rather than exclude each other: vertex displacement carries the coarse relief and changes the silhouette, shadows and depth, while parallax adds the fine detail the tessellation can't carry. A single mode enum (the Unity HDRP shape) makes that combination inexpressible.

material.heightMap = heightTexture;     // shared source, with its existing uv/channel/tiling
material.heightMapBase = 0.5;           // shared zero level: this map value sits at the original surface

material.displacementMode = pc.DISPLACEMENT_VERTEX;  // default DISPLACEMENT_NONE
material.displacementFactor = 0.3;      // amplitude in mesh-local units

material.parallaxMode = pc.PARALLAX_OCCLUSION;  // optional, composes with displacement
material.heightMapFactor = 1;           // parallax depth, unchanged meaning

• The vertex moves along its normal by (height - heightMapBase) * displacementFactor. Sharing the zero level with parallax (Add heightMapBase to unify the parallax zero level across modes #9245) is what makes the features interchangeable representations of the same field: enabling displacement, or toggling parallax on top of it, never shifts the perceived surface. It also covers the offset/boundary control requested in Feature Request - Displacement Map #5554 — the base is the offset, and a negative factor flips the direction.
• displacementFactor is deliberately a different unit from heightMapFactor: local units for geometry, fraction of a UV tile for the parallax lookup. Reusing one factor for both was considered and rejected — they scale different things.
• DISPLACEMENT_NONE/DISPLACEMENT_VERTEX as a string mode rather than a boolean leaves room for a future compute-subdivision mode; neither WebGL nor WebGPU has tessellation stages, so vertex density stays the user's responsibility (or a terrain/LOD system's — that belongs outside the material).

Implementation requirements

Unlike parallax, displacement must run in the vertex stage of every pass — forward, shadow, prepass, picking — or the shadows and depth detach from the surface. That is exactly where the #5573 draft struggled: the shadow/prepass shader variants are built from minimal options that skip every texture map except opacity, so the height map (and its transform/channel plumbing) never reaches those passes today. A real implementation changes that options path, which is the main engine-side work; the chunk itself is small.

• Bounds: the mesh AABB doesn't know about displacement, so culling and shadow casters need padding by |displacementFactor| (same category of problem as morph target bounds). CPU raycasts still see the undisplaced mesh — document, don't solve.
• Normals: the cheap version keeps the original normals and lets the normal map carry the slope, which technically double-counts the low frequencies the geometry now owns. Fine in practice, worth a docs note.

The combined mode

When both features are on, parallax shouldn't re-apply relief the geometry already shows — that double-counts and the surface swims. The fix is cheap:

1. The vertex shader samples the height map at an explicit mip whose texel density roughly matches the vertex density (it needs an explicit LOD anyway — no derivatives in a vertex shader), and passes the sampled height down as a varying.
2. The fragment parallax operates on the residual height(uv) - interpolatedHeight instead of the raw map. The interpolated varying is exactly the low-pass field the geometry realizes, so the residual is high-frequency and near zero mean — and heightMapBase cancels out of it, meaning the combined mode is automatically pivoted on the displaced surface with no extra tuning.

The pleasant emergent property: tessellate more and relief migrates from parallax into real geometry on its own; tessellate less and parallax picks up the slack.

None of this is scheduled — filing it so the design is on record and the 2.22 parallax API can be judged against it.