Add AR Wiener Storm example - #427
Merged
Merged
Conversation
A webcam AR example that pelts the user's MediaPipe-tracked head with floppy, physically simulated raw wieners. Beyond the comedy, it packs several reusable pieces for the webcam AR script family: - tracked-head.mjs: face tracking with the world anchored to the room instead of the head - the camera stays pinned at the origin of MediaPipe's camera space and the tracked head pose drives an entity, so world-simulated physics can be dodged by moving your head. Builds on faceTracking, whose smoothed pose and sim orbit are promoted to protected for subclasses. - head-shadow-catcher.mjs: an invisible head proxy that renders only the shadows cast onto it, written into the transparent canvas alpha as premultiplied black so the darkening lands on the camera feed behind it. Gated by N.L so shadows never wrap past the terminator, and it doubles as a depth occluder. Uses canonical-head.glb, the MediaPipe canonical face mesh extended into a full cranium, so shadows bend around the nose, brow and cheeks. - video-ibl.mjs: low-frequency image-based lighting estimated from the live camera feed, so virtual objects pick up the room's real color and brightness. Falls back to the declared sky when no camera runs. - wiener-storm.mjs: the cannon. Each wiener is a chain of six capsule rigid bodies linked by 6dof spring joints, skinned one-to-one to a six-bone mesh, thrown on drag-aware ballistic arcs aimed at the head at launch. The storm escalates over time and reports hits, dodges and categories as app events for the page HUD (a dignity meter and a deadpan announcer). Physics runs at 180Hz with raised solver iterations (restored on destroy) because head impacts spin the light segments past their joint limits inside a single 60Hz step. Lights are aimed via lookAt plus rotateLocal(90,0,0) since directional lights emit along the entity's negative Y axis. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- video-ibl: enable source mipmaps so the cubemap projection samples each texel once instead of 1024 times, and shrink the atlas to 128px with modest sample counts - the engine's 512px/1024-sample defaults burned tens of milliseconds of GPU per rebuild resolving detail a 64px low-frequency source does not have. - video-ibl: the engine's equirect mapping puts the camera's view direction (-Z) at the panorama seam, not at u=0.5 (+Z), so the observed band was painted 180 degrees behind the camera. Center it on the seam and draw it wrapping across both edges. - head-shadow-catcher: only emit the light0_direction facing gate when a shadow-casting directional light actually joined the catcher layer; with no qualifying light the uniform does not exist and the shader failed to compile. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A webcam AR example that pelts your MediaPipe-tracked head with floppy, physically simulated raw wieners. Whimsical on the surface, but it contributes several reusable pieces to the webcam AR script family.
What's in it
The example (
ar-wiener-storm.html): wieners lob in from the hemisphere between your head and the screen on drag-aware ballistic arcs, bounce off an invisible head proxy, cast real shadows onto your face, vanish behind your head, and can be dodged — or batted away — because the physics world is anchored to the room, not your head. The HUD is a dignity meter (drains per hit, seeps back while you dodge) plus a deadpan announcer, driven entirely by app events.?simruns without a webcam,?debugrenders the head proxy for fit checks.New reusable scripts:
tracked-head.mjs— face tracking with the frame flipped: the camera stays pinned at the origin of MediaPipe's camera space and the tracked head pose drives an entity instead. World-simulated content becomes dodgeable, where the base class's face-locked frame would drag it along with the head. Includes the nose-bridge anchor pin in the flipped frame. Builds onfaceTracking, whose smoothed pose and sim orbit are promoted from private to protected.head-shadow-catcher.mjs— grounds virtual objects on the real head: an invisible proxy that renders only the shadows cast onto it. The stock engine shadow catcher darkens the canvas multiplicatively, which cannot affect a DOM video behind a transparent canvas, so this one routes the engine'sshadowCatcheraccumulator into the canvas alpha as premultiplied black (GLSL + WGSL chunk overrides), gated by N·L so shadows never wrap past the terminator. It doubles as a depth occluder. Accepts any model as the catcher surface, with a cranium ellipsoid fallback.video-ibl.mjs— low-frequency image-based lighting estimated from the live camera feed (in the spirit of mobile AR lighting estimation), so virtual objects pick up the room's real color temperature and brightness. Rebuildsscene.envAtlasinto reused targets on a slow cadence; the declared sky remains the fallback when no camera runs.New assets:
wiener.glb(procedurally authored in Blender: bent capsule, six-bone chain, slick raw-pork material),canonical-head.glb(MediaPipe's canonical face mesh extended into a full cranium, so it registers with the tracked face and shadows bend around the nose and brow — Apache 2.0, sidecar included), andwiener-slap.mp3(a wet slap, velocity-scaled at playback).Reviewer notes
lookAt+rotateLocal(90, 0, 0).pc-entity(model nested as a child) becauseentity:references resolve bare names againstpc-entityelements only; a library-side follow-up for that is tracked separately.face-tracking.mjschanges are doc-visibility only (three members promoted to protected); no behavior change, and all existing examples using it are unaffected.Verified in
?simon both WebGL2 and WebGPU (the catcher's chunk overrides ship in both shader languages); ESLint and the full test suite (1023 tests) pass.🤖 Generated with Claude Code