AppleTV3,2: FairPlay SAP m3 generation is incomplete; mirroring fails with RTSP 466

[3rd3]

Summary

doubletake can pair successfully with an AppleTV3,2 (3rd gen) running AirTunes 220.68, but mirroring does not start because the FairPlay SAP response generated by the current implementation is not accepted by the receiver.

The initial observable failure is:

RTSP/1.0 466 Key Management Error

This is not a network, firewall, or pairing problem. PIN-less legacy pairing works against the receiver, and packet captures from working macOS/iOS senders establish that the FairPlay SAP m3 construction in doubletake is incomplete.

Environment

• Receiver: AppleTV3,2

• Receiver IP: 192.168.178.67

• Receiver server: AirTunes/220.68

• Sender host: Arch Linux

• Command:

  ./doubletake -target 192.168.178.67 -port-range 60000-60010 -debug

• Go version: go1.26.5 linux/amd64

Confirmed Working

• The project builds successfully.

• GStreamer/PipeWire/X11 dependencies are present.

• PIN-less legacy raw pairing succeeds.

• The receiver responds normally to RTSP requests; this is not a connection timeout.

• Successful macOS and iOS captures show that AppleTV3,2 supports FairPlay SAP modes 0, 1, and 3.

• The capture-derived AppleTV3 setup order is:

  pairing
  -> /fp-setup m1
  -> /fp-setup m2
  -> /fp-setup m3
  -> /fp-setup m4
  -> session SETUP
  -> RECORD
  -> compact video SETUP
  

Root Cause

fpsapExchangeM3 currently constructs m3 as:

fixed 144-byte prefix + computed 20-byte tail

However, successful captures show that only the first 16 bytes of m3 are fixed:

46504c59030103000000009803 8f1a9c

The following 128-byte body varies for every successful session, including multiple successful mode-3 sessions from macOS 10.13, current macOS, and iOS.

The final 20-byte tail also varies per session and is not reproduced by the current implementation for known-good captured m2 inputs.

For example, using a successful macOS 10.13 mode-3 exchange:

captured m2:
46504c59030102000000008202037ea24a071259fa83385453c327c55a330d616d760e0650618cfc029637a7ff3797464485bcccad855575828a3d102cee9aaacfb7a7d139912cb28667d6afd0f35c59afb712c137023889df5bad187305bac37b661e366f64103d2be752a896b589cf4f69432ad9e6f000ecd696cb84253848b6144b0e47df69073417cde09b6e

Apple’s successful m3 begins:

46504c590301030000000098038f1a9ca191bf7a4189e7edfb305269acc...

Current fpsapExchangeM3 instead begins:

46504c590301030000000098038f1a9c991ea22c511e45ba97f1af8dfb0f...

Both the 128-byte body and final 20-byte tail differ.

Current Code Findings

• internal/airplay/fpsap.go hard-codes fpsapM3Prefix as 144 bytes.

• internal/airplay/fpsap.go computes only the final 20 bytes from m2.

• internal/airplay/fairplay_message.go originally implemented only the custom FairPlay message decrypt path.

• A matching encrypt/inverse-CBC implementation has now been added and passes all existing modes 0 through 3 round-trip tests.

• Decrypting valid captured mode-3 frames proves that m3 plaintext is not equal to m2 plaintext:

  m2 plaintext begins: 000173c2...
  m3 plaintext begins: 000149b3...
  

  Therefore, m3 requires a separate missing 128-byte plaintext-generation step before it can be encrypted into the m3 body.

Capture Evidence

Working mode-3 senders include:

macOS 10.13.6 / AirPlay/366.76.2
macOS 15.6.1 / AirPlay/870.14.1
iOS 26.5.2 / AirPlay/950.7.1

The AppleTV3,2 accepts all of the following modes in successful captures:

mode 0
mode 1
mode 3

Therefore the receiver does not require mode 0 specifically. The failure is the implementation of m3 generation, not mode selection alone.

Possible fix

1. Replace the fixed fpsapM3Prefix assumption with construction of:
   • fixed 16-byte FPLY/header material;
   • a generated 128-byte mode-specific m3 plaintext;
   • custom FairPlay message encryption;
   • the corresponding mode-specific 20-byte SAP tail.
2. Add captured m2 -> complete m3 golden vectors for modes 0, 1, and 3.
3. Once mode-3 m3 matches the known-good captures, retry against AppleTV3,2.
4. Only after FairPlay succeeds, validate the AppleTV3-specific SETUP -> RECORD -> video SETUP compatibility branch.

Edit by Sol:
The fix criteria should be expanded to require:

• explicit mode extraction and validation;
• independent decrypt and encrypt vectors, not only round trips;
• complete captured m2 → 164-byte m3 vectors;
• captured m3 + ekey → unwrapped key vectors;
• exact commit/version identification;
• an AppleTV3-specific RTSP ordering and plist-shape test;
• not yet demonstrated that the m3 plaintext-generation stage is the only missing stage.

Command outputs

2026/07/19 18:07:38 [HTTP] -> GET /info (body=0 bytes, encrypted=false, cseq=1)
2026/07/19 18:07:38 [HTTP] wrote 79 bytes to socket, waiting for response...
2026/07/19 18:07:38 [READ] reading plaintext response
2026/07/19 18:07:38 [READ] plaintext response header:
RTSP/1.0 200 OK
Content-Length: 648
Content-Type: application/x-apple-binary-plist
Server: AirTunes/220.68
CSeq: 1

2026/07/19 18:07:38 [READ] status=200 content-length=648
2026/07/19 18:07:38 [READ] plaintext body: 648 bytes
2026/07/19 18:07:38 [INFO] full /info response keys: [keepAliveSendStatsAsBody vv audioLatencies keepAliveLowPower macAddress statusFlags audioFormats pk model features name sourceVersion pi deviceID]
2026/07/19 18:07:38 [INFO] audioFormats: [map[audioInputFormats:67108860 audioOutputFormats:67108860 type:100] map[audioInputFormats:67108860 audioOutputFormats:67108860 type:101]]
2026/07/19 18:07:38 [INFO] audioLatencies: [map[audioType:default inputLatencyMicros:0 outputLatencyMicros:0 type:100] map[audioType:default inputLatencyMicros:0 outputLatencyMicros:0 type:101]]
2026/07/19 18:07:38 [INFO] features: 130367356919
2026/07/19 18:07:38 [INFO] statusFlags: 68
2026/07/19 18:07:38 [INFO] keepAliveSendStatsAsBody: true
2026/07/19 18:07:38 connected to: Apple TV (model: AppleTV3,2, initialVolume: 0.0)
2026/07/19 18:07:38 [PAIR] starting transient pair-setup
2026/07/19 18:07:38 [PAIR] trying raw binary pair-setup (UxPlay-compatible)
2026/07/19 18:07:38 [HTTP] -> POST /pair-setup (body=32 bytes, encrypted=false, cseq=2)
2026/07/19 18:07:38 [HTTP] wrote 159 bytes to socket, waiting for response...
2026/07/19 18:07:38 [READ] reading plaintext response
2026/07/19 18:07:38 [READ] plaintext response header:
RTSP/1.0 200 OK
Content-Length: 32
Content-Type: application/octet-stream
Server: AirTunes/220.68
CSeq: 2

2026/07/19 18:07:38 [READ] status=200 content-length=32
2026/07/19 18:07:38 [READ] plaintext body: 32 bytes
2026/07/19 18:07:38 [PAIR] raw pair-setup OK, server Ed25519 pub: 41ee7b4eb658bdbd
2026/07/19 18:07:38 [PAIR] starting raw pair-verify (no HAP encryption)
2026/07/19 18:07:38 [RAW-PV] V1: sending 68 bytes (X25519 pub + Ed25519 pub)
2026/07/19 18:07:38 [RAW-PV] V1 hex: 01000000873adf901dc36c36eb57e6a5df05925c441d4b6b50c8bdb5d7cd0696cf4b2018d73914b9d4b0badb10a99ea493ca41c17ec6307db2ea2bf5e2fd00832dd8fecc
2026/07/19 18:07:38 [RAW] -> POST /pair-verify (body=68 bytes, cseq=3)
2026/07/19 18:07:38 [READ] reading plaintext response
2026/07/19 18:07:38 [READ] plaintext response header:
RTSP/1.0 200 OK
Content-Length: 96
Content-Type: application/octet-stream
Server: AirTunes/220.68
CSeq: 3

2026/07/19 18:07:38 [READ] status=200 content-length=96
2026/07/19 18:07:38 [READ] plaintext body: 96 bytes
2026/07/19 18:07:38 [RAW-PV] V2: received 96 bytes
2026/07/19 18:07:38 [RAW-PV] server signature verified OK
2026/07/19 18:07:38 [RAW-PV] V3: sending encrypted proof (68 bytes)
2026/07/19 18:07:38 [RAW] -> POST /pair-verify (body=68 bytes, cseq=4)
2026/07/19 18:07:38 [READ] reading plaintext response
2026/07/19 18:07:38 [READ] plaintext response header:
RTSP/1.0 200 OK
Content-Length: 0
Content-Type: application/octet-stream
Server: AirTunes/220.68
CSeq: 4

2026/07/19 18:07:38 [READ] status=200 content-length=0
2026/07/19 18:07:38 [RAW-PV] pair-verify complete (connection stays PLAINTEXT)
2026/07/19 18:07:38 [PAIR] raw pair-verify complete (connection stays plaintext)
2026/07/19 18:07:38 pairing complete
2026/07/19 18:07:38 [FP] starting FairPlay SAP handshake...
2026/07/19 18:07:38 [FP] posting m1 (16 bytes) to /fp-setup
2026/07/19 18:07:38 [HTTP] -> POST /fp-setup (body=16 bytes, encrypted=false, cseq=5)
2026/07/19 18:07:38 [HTTP] wrote 157 bytes to socket, waiting for response...
2026/07/19 18:07:38 [READ] reading plaintext response
2026/07/19 18:07:38 [READ] plaintext response header:
RTSP/1.0 200 OK
Content-Length: 142
Content-Type: application/octet-stream
Server: AirTunes/220.68
CSeq: 5

2026/07/19 18:07:38 [READ] status=200 content-length=142
2026/07/19 18:07:38 [READ] plaintext body: 142 bytes
2026/07/19 18:07:38 [FP] received m2 (142 bytes)
2026/07/19 18:07:38 [FP] m2 first 32: 46504c5903010200000000820203e15564080bd22f886a5e730238af78765e53
2026/07/19 18:07:38 [FP] m3 (164 bytes) first 32: 46504c590301030000000098038f1a9c991ea22c511e45ba97f1af8dfb0f86f5
2026/07/19 18:07:38 [FP] posting m3 (164 bytes) to /fp-setup
2026/07/19 18:07:38 [HTTP] -> POST /fp-setup (body=164 bytes, encrypted=false, cseq=6)
2026/07/19 18:07:38 [HTTP] wrote 306 bytes to socket, waiting for response...
2026/07/19 18:07:38 [READ] reading plaintext response
2026/07/19 18:07:38 [READ] plaintext response header:
RTSP/1.0 200 OK
Content-Length: 32
Content-Type: application/octet-stream
Server: AirTunes/220.68
CSeq: 6

2026/07/19 18:07:38 [READ] status=200 content-length=32
2026/07/19 18:07:38 [READ] plaintext body: 32 bytes
2026/07/19 18:07:38 [FP] received m4 (32 bytes)
2026/07/19 18:07:38 [FP] m4 payload (20 bytes): 285b537aa17d619923591bc3f508bc398c270323
2026/07/19 18:07:38 [FP] ekey chunk1 [16:32]: a8716bf5a8ebd2704e9d9a2ef0ea1352
2026/07/19 18:07:38 [FP] ekey chunk2 [56:72]: c24e9d42bb7eb4ec22560ed1ed0a2e67
2026/07/19 18:07:38 [FP] unwrapFairPlayKey fpAesKey: a82e9dfd42417aa4ddc4f68005988778
2026/07/19 18:07:38 [FP] m3 first 32 bytes: 46504c590301030000000098038f1a9c991ea22c511e45ba97f1af8dfb0f86f5
2026/07/19 18:07:38 [FP] hashed with SharedSecret (32 bytes)
2026/07/19 18:07:38 [FP] FairPlay SAP handshake complete!
2026/07/19 18:07:38 [FP] fpAesKey (raw): a82e9dfd42417aa4ddc4f68005988778
2026/07/19 18:07:38 [FP] fpKey (hashed): 32c701689c4cfc71c72b8d31dea0db68
2026/07/19 18:07:38 [FP] stream IV:      555c1792573d9c578ce7ebef5f191b86
2026/07/19 18:07:38 FairPlay setup complete
2026/07/19 18:07:38 [SETUP] using encryption (key: 16 bytes, IV: 16 bytes)
2026/07/19 18:07:38 [SETUP] consecutive UDP ports: timing=60000 ctrl=60001 data=60002
2026/07/19 18:07:38 [SETUP] event listener on TCP port 60003
2026/07/19 18:07:38 [SETUP] audio encryption: AES-128-CBC (fpKey/hashed 16 bytes)
2026/07/19 18:07:38 [SETUP] FairPlay ekey=72 bytes, eiv=16 bytes, et=32
2026/07/19 18:07:38 [SETUP] phase 1 (audio+session): ct=2 spf=352 audioFormat=0x40000 controlPort=60001
2026/07/19 18:07:38 [RTSP] -> SETUP rtsp://192.168.178.67:7000/1784477258151618612 (body=674 bytes, encrypted=false, cseq=7)
2026/07/19 18:07:38 [RTSP] wrote 846 bytes to socket, waiting for response...
2026/07/19 18:07:38 [READ] reading plaintext response
2026/07/19 18:07:38 [READ] plaintext response header:
RTSP/1.0 466 Key Management Error
Content-Length: 0
Server: AirTunes/220.68
CSeq: 7

2026/07/19 18:07:38 [READ] status=466 content-length=0
2026/07/19 18:07:38 [READ] error response body (0 bytes): 
2026/07/19 18:07:38 mirror setup failed: SETUP phase 1 (audio): HTTP 466 (body: )

[omarroth]

Thanks for the info! I'm not able to reproduce this locally, but would you be able to verify whether e544a88 fixes this issue?

[3rd3]

I tested commit e544a88 against the same AppleTV3,2 (AirTunes/220.68).

It fixes the original RTSP/1.0 466 Key Management Error: FairPlay setup now completes and the receiver accepts the subsequent audio/session SETUP.

The session now gets much further:

• Raw pair setup and pair verify succeed.
• FairPlay /fp-setup m1 through m4 succeed.
• Audio/session SETUP succeeds.
• Video SETUP succeeds.
• The Apple TV connects back to the advertised event and video data ports.
• RECORD and SET_PARAMETER return 200 OK.
• NTP timing packets are exchanged.
• doubletake sends the codec configuration, IDR frames, and P-frames.

However, the Apple TV only turns black and remains black.

I also ruled out two capture-side suspects:

• XFCE/X11 capture now forces I420 (gstArgs "video/x-raw,format=I420"), so the stream is standard H.264 High Profile rather than High 4:4:4 Predictive.
• Changing the desktop to 1920x1080 produces a 1920x1080 H.264 High Profile Level 4.0 stream, but it remains black.

So e544a88 is a real improvement: it resolves the FairPlay/key-management rejection on this receiver. The remaining AppleTV3,2 issue appears to be after RTSP setup, likely in video data-channel compatibility such as packet framing, timestamps, stream key derivation/AES-CTR encryption, or another AppleTV3-specific mirroring detail.

[3rd3]

I managed to get it running by changing the AES-CTR key and IV derivation. But it only works with <=720p seemingly because the resolution is limited to that on this model/stream configuration. Above 1080p I got a black screen. At 1080p the video was cropped to 720p. When I fixed that by forcing the receiverPresentationSize to 720p, the downscaling would result in frequent congestion. But streaming at 720p screen resolution works well with only occasional, minor congestion (every ~10 minutes). Though this may have been due to network limitations unrelated to doubletake.

[objevovat]

I think I can explain why @3rd3's fix works and why it hasn't been safe to merge as-is. I don't have an AppleTV3,2, so this is code reading rather than a test — but it's specific enough to check.

The mechanism

ekey wraps the raw fpAesKey. That's the only key material the receiver ever gets, so it's the only key it can end up holding after unwrapping.

But fairplay.go:113-124 doesn't hand that key to the stream cipher:

finalKey := c.fpAesKey
if c.PairKeys != nil && len(c.PairKeys.SharedSecret) > 0 {
    h := sha512.New()
    h.Write(c.fpAesKey)
    h.Write(c.PairKeys.SharedSecret)
    finalKey = h.Sum(nil)[:16]   // fairplay.go:118
}
c.fpKey = finalKey

and mirror.go:64 (encKey := c.fpKey) and mirror.go:173 (audioKey = c.fpKey) both use that mixed value.

Why that breaks the legacy path specifically

The condition is len(SharedSecret) > 0 — but the legacy PIN-less path sets a shared secret too.

performTransientSetupAndVerify (pairing.go:96) falls through to rawPairVerify when raw pair-setup succeeds (pairing.go:125), and rawPairVerify ends with c.PairKeys.SharedSecret = shared at pairing.go:688 — right under the comment "do NOT enable HAP encryption on the control channel".

So on an AppleTV3,2 the mixing fires even though nothing in that handshake tells the receiver to do the same. The sender encrypts with SHA-512(fpAesKey ‖ shared)[:16], the receiver decrypts with fpAesKey. Every frame decodes to noise, which matches the symptom exactly: RTSP setup succeeds, RECORD returns 200, frames flow, screen stays black.

Why the fork patch isn't safe to merge unchanged

74563a4d changes encKey := c.fpKey to encKey := c.fpAesKey unconditionally. That's right for AppleTV3, but it would remove the mixing on genuine HAP-paired AirPlay 2 receivers, where the comment at fairplay.go:115 describes what the receiver actually does:

The receiver does: SHA-512(fairplay_decrypt(ekey) || ecdh_secret)[:16]

So the current code isn't wrong in general — it's wrong about when to apply the mixing.

The distinction already exists in the code

c.encrypted is set to true only on the HAP path (pairing.go:488, inside PairVerify). rawPairVerify deliberately leaves it false. So the condition at fairplay.go:114 can key off that instead of off whether a shared secret happens to be present:

if c.encrypted && c.PairKeys != nil && len(c.PairKeys.SharedSecret) > 0 {

That should leave AirPlay 2 receivers on exactly the bytes they get today, while giving the legacy path the raw fpAesKey that ekey actually wrapped — the same key 74563a4 arrives at, but only where it belongs.

Caveats

• Untested. No AppleTV3,2 here. @3rd3, if you still have that setup, this one-line change instead of the fork patch would be the thing to try.
• It only covers the key derivation. The resolution/receiverPresentationSize half of 74563a4 is a separate problem and this says nothing about it.
• If a HAP-paired receiver ever also needs the raw key, c.encrypted is the wrong discriminator and the real signal is the security mode negotiated at SETUP.

For what it's worth, I arrived at this from the FairPlay side — I have an independent implementation of the SAP exchange, and it agrees with yours on 151 payloads (#22), so the m3 half looks settled. This is downstream of that.

[3rd3]

@objevovat I just confirmed this one-line change works. However, in capture.go:224 it was also necessary to add "!", "video/x-raw,format=I420".

[objevovat]

Thanks for testing it @3rd3 — that's the confirmation I couldn't get.

Opened #23 with the change plus tests. It extracts the derivation into deriveStreamMasterKey so the four legacy/HAP combinations can be checked without a handshake, and adds a guard that fails if rawPairVerify ever starts setting c.encrypted — otherwise legacy receivers would silently regress to the mixed key and this would come back as another black screen.

I left the capture.go I420 change out. It looks like a separate capture-side problem and may already be covered by e9a395a; I can't test a GStreamer pipeline against an Apple TV, so guessing at it seemed worse than saying so. Probably worth its own issue if it's still needed on current main.

One correction to my earlier comment: I said I had an independent implementation of the SAP exchange. That was true then — it's what produced the vectors in #22 — but I've since replaced it with this project's closed form, since yours is smaller and faster than what I had. So that agreement is a snapshot from when the two were genuinely separate, not something I can re-derive now.

[3rd3]

I've confirmed that e9a395a fixes the format issue.

The only remaining issue then seems to be dealing with unsupported screen resolutions.

[3rd3]

@omarroth Just noticed that AppleTV3,2 is actually AirPlay 1 (since you only mention 2 in the README). Just a minor detail.

[omarroth]

Updated README, appreciate it! Last piece should be fixed with #25.