Skip to content

feat(receiver): add validated native Metal renderer - #174

Draft
Betafer wants to merge 2 commits into
swellweb:maint-3.5from
Betafer:codex/feat-native-metal-renderer-v3.5
Draft

feat(receiver): add validated native Metal renderer#174
Betafer wants to merge 2 commits into
swellweb:maint-3.5from
Betafer:codex/feat-native-metal-renderer-v3.5

Conversation

@Betafer

@Betafer Betafer commented Aug 21, 2026

Copy link
Copy Markdown

Summary

Add an optional native CAMetalLayer renderer for VideoToolbox NV12 frames, while retaining the existing SDL/OpenGL renderer as the default and immediate fallback.

The native path wraps the decoder CVPixelBuffer planes through CVMetalTextureCache, avoiding the full-frame CPU upload performed by SDL_UpdateNVTexture. It also renders the remote cursor in the same Metal pass, preserves the display's Display P3 color space and accounts for Retina backing scale.

Safety

  • OpenGL remains the default backend
  • native Metal is enabled only through TB_RECEIVER_RENDER_DRIVER=metal-native or auto
  • auto mode evaluates real submitted, completed and dropped frames before keeping Metal
  • renderer creation or frame submission failure immediately exposes the existing OpenGL path
  • software-decoded/raw frames continue through OpenGL
  • the current normal-size macOS cursor behavior is preserved

Structure

  1. add the opt-in zero-copy renderer and focused bridge test
  2. add adaptive health policy, Display P3 handling and reproducible Metal/OpenGL benchmarks

Verification

  • rebased cleanly onto maint-3.5 at v3.5.1-rc.3
  • Receiver make test: 664 logic checks pass; the platform test builds successfully
  • full Receiver build succeeds
  • Apple M4, 4096 x 2304 at 60 fps: Metal 180/180 frames, 0 drops, 0.223 ms average submission, 1.306 ms average GPU time, Display P3
  • same run with SDL/OpenGL: 180/180 frames, 7.553 ms average frame call, 2430 MiB CPU upload in three seconds
  • extended 2017 Intel Retina iMac test: 600/600 Metal frames, 0 drops, hardware VideoToolbox HEVC and Display P3

This is intentionally a Draft PR because the production change is larger than the maintenance fixes and benefits from maintainer review before enabling it outside an opt-in setting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant