Skip to content

refactor: compile privacy policies onto rrweb's own masking primitives (Privacy at Capture v2) - #2

Closed
roggernaut wants to merge 24 commits into
mainfrom
privacy-v2-simplification
Closed

roggernaut wants to merge 24 commits into
mainfrom
privacy-v2-simplification

Conversation

@roggernaut

@roggernaut roggernaut commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Summary

Reworks Privacy at Capture (#1) so it matches field-proven practice from the five major session-replay vendors (PostHog, Highlight, Sentry, Amplitude, Mixpanel), resolving all 24 findings from the post-merge review. Design spec and prior-art appendix: docs/superpowers/specs/2026-08-25-privacy-v2-simplification-design.md.

Architecture

  • compilePrivacyPolicy now compiles presets/rules onto rrweb's existing masking primitives (selector lists + inherited needsMask propagation). The parallel getPrivacyAction per-node rule engine is deleted (privacy.ts: 936 → ~400 lines).
  • legacy (no policy, no plugin) is inert: the perf smoke test asserts zero privacy selector matching on the default path.
  • New unmaskTextSelector (record option + allow/unmask rules), nearest-ancestor-wins, with a presence probe restoring subtree short-circuiting when nothing on the page can unmask.
  • Cross-vendor mask/unmask/block classes (.rr-mask, .mp-mask, .fs-mask, .amp-mask, .ph-mask, .sentry-mask, …) recognized under non-legacy presets.

Fail-closed hardening

  • Per-selector validation at compile (invalid selectors dropped with a warning — one bad selector can no longer poison the merged block list, which previously failed open).
  • Mask decisions catch-to-mask; sanitizeUrl strips username:password@ and fails closed on unparseable URLs; malformed plugin-transformed policies fall back to the user's policy instead of crashing record().
  • Protected inputs (password/hidden/cc-*/OTP autocomplete) always masked; maskInputFn/maskAttributeFn outputs are constrained under presets (fn controls length, policy is final authority); one attribute-finalization pass covers snapshot, mutation, and added nodes.
  • Canvas masking forces the FPS capture path (the unmasked mutation-mode command stream is unreachable); mask regions use content-box math.
  • Detectors: fixed whole-value set (email, phone, Luhn card, SSN, IPv4) — custom regex patterns removed (bypassable ReDoS validator with them); detection covers text nodes, live text mutations, and input values, and works under any preset when the plugin is loaded.
  • CSS is never masked, on any path (unanimous vendor precedent).

Breaking changes

See .changeset/privacy-v2-simplification.md: @rrweb/types major (union worker params, removed policy fields), needMaskingText signature, whole-value detector semantics, mask-fn composition rules, protected-inputs default, invalid-selector fail-closed.

Review provenance

11 planned tasks, each implemented and reviewed by independent agents with fix rounds (9 Important + 1 Critical finding fixed pre-merge across rounds), plus a whole-branch final review ("merge with fixes" → fix wave verified). Known post-merge backlog (probe caching, srcset sanitization, attribute-sweep early-out, mutation-path unmask e2e coverage, maskInputValue deprecation) is listed in the final review triage.

Test plan

  • rrweb-snapshot: 199 passed / 1 skipped (unit + integration + perf smoke)
  • rrweb: full suite green modulo two known Puppeteer load flakes (pass in isolation)
  • plugin: 4/4; tsc -b clean across the monorepo
  • Empirical regex verification (phone formats, adjacency case, ReDoS shapes)

🤖 Generated with Claude Code

Provenance and production mileage

A source-level audit classified every mechanism on this branch against the five vendor codebases (PostHog, Highlight, Sentry, Amplitude, Mixpanel forks/SDKs). Full classification in the review record; summary:

Vendor-proven (a vendor runs essentially the same mechanism in production): selector-list compilation and union merging (Sentry getPrivacyOptions); per-selector validation with drop+warn (Amplitude, near-verbatim); cross-vendor mask/block class recognition (Mixpanel); the input-masking composition core incl. star-over-fn (Sentry) and always-masked protected inputs (Mixpanel/Amplitude/Sentry); the attribute finalization sweep, late-write guard, generated-attribute exemption, and maskAll-over-fn precedence (PostHog fork, near-verbatim); the title/placeholder/aria-label masked-attribute defaults (Sentry); canvas mask regions, worker fillRect, and snapshot suppression (PostHog fork); whole-node text-detector masking on snapshot AND live characterData mutations (Highlight fork).

Inspired (concept proven, implementation ours): the preset/rules policy translation itself; unmask-in-the-walk with nearest-ancestor semantics; '*'-as-fallback demotion; SSN/card/email/IPv4 patterns (derived with corrections); data-privacy enumerated attribute; unconditional catch-to-mask (stricter than every vendor); untaintedTagName (proven accessor machinery, new target); strict media-source attribute nulling (Highlight does element-level).

Invented here — no production mileage anywhere (review these hardest):

  • DOM-attribute URL sanitization in its entirety (no vendor sanitizes URLs in the recorded DOM): strict-masks-all-params, userinfo stripping, synthetic-base relative round-trip, fail-closed ''
  • The unmask-selector presence probe (resolveUnmaskTextSelector, shadow-recursive, per snapshot/flush) and the mask-all selector cache
  • The phone pattern + digit-count gate; the MAX_SCAN_LENGTH mask-everything cliff
  • Detector scanning of input values (incl. per-keystroke) — Highlight scans text nodes only
  • maskAttributeFn as a pipeline stage (vendors treat it as terminal), fn try/catch-to-stars, non-string-to-stars
  • The versioned policy schema itself; the plugin applyPrivacyPolicy hook and its compile fallback
  • resolveCanvasSampling coercion; content-box mask scaling; the tracked-shadow-roots canvas registry
  • FORM_VALUE_TAGS gating of mutation value masking; rr_src through URL/media policy; strict forcing recordCanvas off

Rogier Trimpe and others added 24 commits August 25, 2026 21:57
Design for reworking the merged privacy feature onto rrweb's existing
masking primitives, adopting field-proven mechanisms from PostHog,
Highlight, Sentry, Amplitude, and Mixpanel. Resolves all 24 confirmed
review findings structurally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the rule-engine CompiledPrivacyPolicy with a v2 shape that
compiles a PrivacyPolicy into plain rrweb masking options (selector
lists, maskAllInputs, maskedAttributes, blockMedia, sanitizeUrls,
precomputed query-parameter sets). Presets are now strict/balanced/legacy
only ('custom' removed), actions drop style/classification, and
'unmask' is an alias of 'allow'. Vendor privacy classes (rr-/mp-/fs-/amp-/
ph-/sentry-) are compiled into the mask/unmask/block selectors for every
non-legacy preset.

Delete the old rule-matching and heuristic-detector-compilation engine
(getPrivacyAction, maskTextWithPrivacy, maskInputWithPrivacy,
maskAttributeWithPrivacy, protectSerializedAttribute,
detectSensitiveText, and related helpers) along with the JSON policy
schema; detector compilation and masking behavior land in later tasks.
snapshot.ts and the rrweb recorder fall back to their pre-privacy
masking paths in the interim (plain needsMask/maskInputValue/attribute
passthrough); sanitizeUrl is a passthrough stub pending Task 3.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n masking

Task 1's deletion of protectSerializedAttribute silently stopped honoring
the public maskAllElementAttributes/maskAttributeFn recording options,
since nothing consumed them after the deletion even though snapshot.ts
and MutationBuffer kept threading them through. Restore a minimal final
sweep over string attribute values in both serializeElementNode (full
snapshots) and MutationBuffer's attribute-mutation emit path (masking
takes precedence over the compact-style-mutation optimization, which
would otherwise leak unmasked style fragments through a styleDiff
object). Both are marked for replacement by Task 6's finalizeAttribute.

Also harden isProtectedInput to fail closed (treat as protected) when
tagName is shadowed to a non-string value, instead of silently treating
it as not an input.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…attern machinery

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ics, add spaced/dashed formats

Fixes two issues from review:
1. Phone pattern quantifier changed from 7-18 to 7-13 to prevent matching across
   digit runs separated by spaces (e.g. '5551234567 4111 1111' no longer matches as
   single run). Restores space in character class [\d ().-] to support spaced
   formats ('555 123 4567'), dashed ('555-123-4567'), and mixed formatting.
2. Restore 4 files accidentally committed in prior commit from 54798a2:
   - packages/plugins/rrweb-plugin-network-record/tsconfig.json
   - packages/plugins/rrweb-plugin-network-replay/tsconfig.json
   - packages/plugins/rrweb-plugin-privacy-detectors/tsconfig.json (tsconfig reformatting)
   - packages/rrweb-player/.svelte-kit/ambient.d.ts (SvelteKit machine-local regeneration)

Adds test assertions for spaced and dashed phone detection to prevent future regression.

Tool note: tsconfig.json files are auto-reformatted by TypeScript build processes;
.svelte-kit/ambient.d.ts is regenerated by SvelteKit and embeds local environment
variables - both must be kept out of version control.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ils closed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…CSS exempt everywhere

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… exempt <style> text mutations

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e, CSS attrs exempt

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pipeline stage

rr_src (the name a cross-origin iframe src is renamed to before finalization)
was in neither URL_ATTRIBUTES nor MEDIA_SOURCE_ATTRIBUTES, so strict did not
null it and balanced did not sanitize it -- userinfo and query tokens survived
verbatim. Add it to both sets, keeping the rename-then-finalize order.

maskAttributeFn no longer returns early: its output feeds into the policy
block, which stays the final authority (design spec 5). Under legacy that
block is the identity, so callback output survives verbatim; under
balanced/strict the policy applies on top and can only narrow it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The empty-string guard added in the previous commit returned before the whole
policy block, so a maskAttributeFn returning '' bypassed the strict
media-source null branch. rebuild.ts distinguishes null (attribute removed)
from '' (setAttribute(name, '')), so a strict <img src>/<iframe src> emptied by
a callback replayed as src="" instead of a dropped attribute.

Delete the guard and handle the narrow case it protected against inside
sanitizeUrl, which now returns '' for '' instead of resolving it to '/'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Task 1 already removed the CSS masking paths in observer.ts/mutation.ts;
the only remnant was StylesheetManager's maskAdoptedRule stub and its
now-unused privacy constructor param. Deletes both and flips the
stylesheet-manager.test.ts assertion to the v2 invariant: adopted-sheet
rules are recorded verbatim regardless of privacy policy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ing; cheap canvas discovery

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…in detects under legacy

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ntedTagName, close form-shadowing crash

A <form> with a descendant control named "tagName" shadows the tagName
getter; toLowerCase(target.tagName) in the attributes-mutation branch
crashed on it uncaught inside the MutationObserver callback. Route the
remaining reads through untaintedTagName (or the existing targetTagName)
and add a regression test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mirror rrweb-snapshot's v2 privacy types into @rrweb/types, expose the
missing unmaskTextSelector as a record() option (it was previously hardcoded
to null when merging), rewrite guide.md's Privacy section to describe v2
exactly, and add the consolidated privacy-v2-simplification changeset.

Also corrects the privacy-detectors plugin README, which claimed detection
covers input values -- it currently only scans page text nodes at snapshot
time.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The approved design (spec Decisions + §6) says a detector hit masks the
whole text node / input value, but only serializeTextNode was wired up:
input values (snapshot and live input events) and characterData mutations
bypassed detection entirely. Route maskInput and the mutation-buffer
characterData path through detectSensitiveValue, mirroring the text-node
hook's rule that detectors only inspect values that would otherwise leave
unmasked (a trusted legacy maskTextFn/maskInputFn composition is kept, and
<style>/<script> text is never scanned).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… footnote

Round-1 review fixes:
- guide.md's heuristic-detection paragraph claimed detectors only scan text
  nodes at snapshot time. That was true when written, but a concurrent fix
  (f34e6c7) wired detectSensitiveValue into maskInput() and the
  characterData mutation path, so detectors now also cover input values and
  live text-mutation updates (verified via `grep -rn "detectSensitiveValue("`
  across packages/rrweb-snapshot/src and packages/rrweb/src: three call
  sites -- snapshot.ts:622, utils.ts's maskInput, mutation.ts's characterData
  case -- vs. finalizeAttribute, which has none, so attribute values are
  still not scanned). Updated the paragraph to match.
- Reconciled .changeset/calm-ravens-protect.md, which still described the
  pre-simplification design (a 'custom' preset, "data-privacy works without
  recorder-specific configuration") and now contradicts the shipped v2
  behavior; left kind-pumas-detect.md alone (already accurate).
- Deleted .changeset/khaki-hoops-smile.md, an empty/broken stub (`---\n---`,
  no version bumps, no body) and .changeset/loud-lions-protect.md, which
  claimed policies "apply to CSS text" -- the opposite of the final "CSS is
  never masked" behavior.
- Added a guide.md footnote documenting the legacy-preset corner case: a
  selector-based mask/exclude rule also activates the corresponding
  [data-privacy="mask"/"exclude"] recognition under legacy, but
  data-privacy="allow" is never recognized under legacy regardless.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…walk short-circuit, blockMedia

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reword explicit vendor-name references in shipped code and docs to
'inspired by' phrasing; the mechanisms were re-implemented for rrweb, not
ported. Remove the internal design/plan documents from the branch — they
are working artifacts, not upstream deliverables.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@roggernaut

Copy link
Copy Markdown
Owner Author

Superseded by #3 (renamed to the standard's name, Privacy at Capture, and carrying the post-audit hardening). The stacked merge set — privacy-at-capture-core plus three experimental layers — will follow from #3.

@roggernaut roggernaut closed this Aug 26, 2026
@roggernaut
roggernaut deleted the privacy-v2-simplification branch August 26, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant