Repository navigation
Conversation
Replace the legacy, frozen @mediapipe/face_mesh and @mediapipe/holistic
Solutions API (CDN-dependent, ~41MB vendored) with the actively-maintained
Tasks Vision API (FaceLandmarker/HolisticLandmarker), fully self-hosted
under vision/ (~29MB: shared SIMD wasm runtime + two .task models).
- vision/worker.js: unified multi-thread worker, role selected via
Worker({name: "face"|"holistic"}), adapts Tasks Vision's per-detection
array results back into the flat shape ml-manager.js/converter/*.js
already expect, so those files need no changes.
- vision/single.js: equivalent main-thread wrapper for the single-thread
(MULTI_THREAD=false) path.
- landmark-manager.js: model loading is now lazy per tracking mode/thread
setting instead of eagerly loading all three model runners at startup.
- Fix a latent bug only reachable via the new main-thread path: the
debug canvas's id="dbg" created an implicit window.dbg global that
collided with Emscripten's internal debug-logging hook, renamed to
"dbgcanvas".
- health-manager.js: don't evaluate/raise FPS alerts before startup has
finished (LOADING_SCENE) or while the tab is backgrounded
(document.hidden), both of which were legitimately near-zero FPS and
triggered false "Hardware Acceleration"/"Full Screen" alerts.
- media-manager.js: pause the camera on tab hide, resume on tab show
(unless the user has manually toggled the camera off or is using
iFacialMocap instead).
- control-manager.js: drop the dead openlive3d_s3_put_log API call
(endpoint no longer functioning).
- vrm-manager.js: clamp the VRM load-progress log to 100% (compressed
Content-Length vs decompressed loaded-bytes could exceed 100%).
Config version bumped to v.2.7.0 (2026-09-20).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LcordyKhiwp7W7HMWTdX3F
Revert the id="dbg" -> "dbgcanvas" rename (gui-layout.js): it fixed the collision but only for consumers whose HTML uses that exact id, creating a hidden coupling between this shared submodule and every consuming page's markup (and an inconsistency risk between the website and the Electron app, which both happen to use id="dbg"). Instead, shadow the ambient global at the source: vision/single.js now declares `let dbg;` at module scope. Classic <script> tags on a page share one global lexical scope, and a lexical binding always takes precedence over window's own/named properties for a bare identifier lookup - so this correctly neutralizes Emscripten's `typeof dbg` guard for any consuming page, regardless of what it names its debug canvas, with no HTML changes required in either consumer. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcordyKhiwp7W7HMWTdX3F
Ghostkwebb
reviewed
Sep 20, 2026
|
|
||
| constructor(cb) { | ||
| this.cb = cb; | ||
| this.fLandmarkerReady = FilesetResolver.forVisionTasks("ol3dc/vision/wasm").then((filesetResolver) => { |
Contributor
There was a problem hiding this comment.
Might want to use getCMV("INTEGRATION_SUBMODULE_PATH") instead of "ol3dc" here so the web/core version doesn't 404. Hmmmmmmmmmmmmmmmmmmmmm
Ghostkwebb
reviewed
Sep 20, 2026
| } | ||
|
|
||
| onmessage = async e => { | ||
| if (landmarker && e.data && e.data["metakey"] && e.data["image"]) { |
Contributor
There was a problem hiding this comment.
When switching tracking modes, this drops incoming frames while the model loads and freezes the tracking loop. Can we await landmarkerReady here instead, like in single.js?
Ghostkwebb
reviewed
Sep 20, 2026
| metakey = e.data["metakey"]; | ||
| try { | ||
| let raw = await landmarker.detect(e.data["image"]); | ||
| postMessage({ |
Contributor
There was a problem hiding this comment.
Should we call e.data.image.close() in a finally block here so transferred ImageBitmap frames don't leak GPU memory during long streams?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the legacy, frozen @mediapipe/face_mesh and @mediapipe/holistic Solutions API (CDN-dependent, ~41MB vendored) with the actively-maintained Tasks Vision API (FaceLandmarker/HolisticLandmarker), fully self-hosted under vision/ (~29MB: shared SIMD wasm runtime + two .task models).
Config version bumped to v.2.7.0 (2026-09-20).
Claude-Session: https://claude.ai/code/session_01LcordyKhiwp7W7HMWTdX3F