diff --git a/README.md b/README.md index aec7de1..daa441a 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ [![node](https://img.shields.io/node/v/broomsticks.svg)](https://nodejs.org) [![license](https://img.shields.io/npm/l/broomsticks.svg)](./LICENSE) -> **Status: early access.** The npm name is reserved and the design is locked (see [PLAN.md](./PLAN.md)); the scanner is in active development. The published `broom` command currently prints a notice only — it reads, writes, and transmits nothing until the engine lands. Watch the repo for `v0.1`. +> **Status: early access.** The scanner, redactor, source adapters (Claude Code, Codex, Cursor), allowlist, Claude Code hook installer, and the live redacting proxy are all implemented and covered by tests. Being finalized for the `v0.1` npm release — until then, install from git. Watch the repo. --- @@ -20,32 +20,40 @@ AI coding assistants keep a full, local, plaintext record of every session — y Those transcripts then live on disk indefinitely — `~/.claude/projects/**/*.jsonl`, `~/.codex/*.jsonl`, Cursor's `state.vscdb` SQLite stores — and get swept into Time Machine, Dropbox, `rsync` backups, or a shared machine. A leaked key in a transcript is just as live as one in a committed `.env`, but nothing scans for it. -**broomsticks** is a small, auditable CLI that finds those secrets and scrubs them out of the transcripts in place — safely. +**broomsticks** gives you two complementary tools: + +1. **`broom scan` / `broom clean`** — find secrets already written to your transcripts and scrub them out in place, safely. +2. **`broom proxy`** — a local redacting proxy that sits between your AI tools and the provider API, stripping secrets out of requests *before they're ever sent* (and out of responses the model echoes back). ## Design principles -- **Dry-run by default.** Nothing is ever modified unless you pass `--apply`. +- **Dry-run by default.** `clean` never modifies anything unless you pass `--apply`. - **Back up before every write.** Each touched file is copied to a timestamped backup directory first. -- **Transcripts only.** It never touches credential files themselves (`~/.codex/auth.json`, `~/.aws/credentials`, etc.) — only chat history. +- **Transcripts only.** The scanner never touches credential files themselves (`~/.codex/auth.json`, `~/.aws/credentials`, etc.) — only chat history. +- **Local by default.** `scan`/`clean` do no networking at all (enforced by a test). Only `broom proxy` talks to the network — by design, since it *is* the network path. - **Auditable supply chain.** Plain ESM JavaScript, **zero runtime dependencies**, no build step. The code published to npm *is* the source — read every line before you trust it with your secrets. - **Non-reversible, idempotent redaction.** Secrets are replaced with `«BROOM::»`. The hash lets you correlate without leaking, and re-running never double-redacts. ## Install ```bash -# one-off, no install +# one-off, no install (once published) npx broomsticks scan # or install the CLI globally npm install -g broomsticks broom scan + +# from git, before the npm release +git clone https://github.com/digitaldrreamer/broomsticks +npm install -g ./broomsticks ``` -Requires **Node ≥ 22.5** (uses the built-in `node:sqlite` to read Cursor's database — no native modules). +Requires **Node ≥ 22.13** — it uses the built-in `node:sqlite` to read Cursor's database (no native modules), which is available without the `--experimental-sqlite` flag as of Node 22.13.0. Prefer no Node at all? A dependency-light **`scripts/broom.sh`** fallback (using `jq` + the `sqlite3` CLI) is planned for environments where you can't or won't run the package. -## Usage (target CLI) +## Clean up: `scan` / `clean` ```bash # Scan every supported source, print a redacted report. Exits non-zero if anything is found (CI-friendly). @@ -55,6 +63,9 @@ broom scan broom scan --source claude-code broom scan --source cursor +# List discovered transcript files across all sources +broom sources + # Preview what would change, without writing broom clean @@ -71,13 +82,77 @@ broom clean --apply --extra ./leaked.txt | Flag | Meaning | | --- | --- | | `--source ` | Restrict to `claude-code`, `codex`, or `cursor` (repeatable; default: all) | -| `--apply` | Perform redaction (otherwise dry-run) | +| `--apply` | Perform redaction (`clean` only; otherwise dry-run) | | `--backup-dir ` | Where backups go (default `~/.broom/backups//`) | | `--no-backup` | Skip backups (discouraged) | -| `--extra ` | Additional literal/regex secrets to redact | +| `--extra ` | Additional literal/regex secrets to redact (one per line) | +| `--allowlist ` | Custom allowlist file (default `~/.broom/allowlist.txt`) | +| `--no-allowlist` | Disable allowlist suppression — report every finding | | `--json` | Emit findings as JSON | | `--no-fail` | Exit `0` even when secrets are found | +## Prevent: `broom proxy` + +Cleaning up after the fact only reduces exposure — the secret still reached the model. The proxy stops the leak at the source. It sits in front of the provider API and redacts secrets out of the text and tool-call content in every outgoing request before it's sent, and out of every response the model echoes back. + +```bash +# Start the proxy in the foreground (Ctrl-C to stop) +broom proxy + +# Then point your AI tools at it: +export ANTHROPIC_BASE_URL=http://127.0.0.1:7777 +export OPENAI_BASE_URL=http://127.0.0.1:7777 +``` + +| Route | Upstream | Used by | +| --- | --- | --- | +| `POST /v1/messages` | `api.anthropic.com` | Claude Code, Aider | +| `POST /v1/chat/completions` | `api.openai.com` | Codex, OpenAI-compatible clients | + +Both streaming (SSE) and non-streaming responses are handled — a streaming response is buffered in full before redaction so a secret straddling two chunks can't slip through, then re-emitted as a valid SSE stream. Redaction covers both assistant **text** and **tool-call arguments** (e.g. a secret the model echoes into a `write_file` content or shell-command argument). Extended-thinking blocks are the one exception — see [Limitations](#limitations). + +Make it permanent instead of exporting vars by hand: + +```bash +# Add ANTHROPIC_BASE_URL / OPENAI_BASE_URL to your shell init files +broom proxy --install + +# Also register a login-persistent daemon (launchd on macOS, systemd --user on Linux) +broom proxy --install --daemon + +# Remove the daemon later +broom proxy --uninstall +``` + +> **Note:** `--uninstall` removes both the daemon **and** the `ANTHROPIC_BASE_URL` / `OPENAI_BASE_URL` block it added to your shell profiles. Open a new terminal (or re-source your profile) afterward so your shell stops pointing at the now-stopped proxy. + +| Flag | Meaning | +| --- | --- | +| `--port ` | Port to listen on (default `7777`) | +| `--verbose` | Log redaction counts to stderr | +| `--allowlist ` | Allowlist to suppress known false positives | +| `--install` | Add base-URL env vars to your shell init files | +| `--install --daemon` | Also install a login-persistent daemon (logs to `~/.broom/proxy.log`) | +| `--uninstall` | Remove the daemon | + +## Automate: `broom install` + +For Claude Code users, `broom install` wires broomsticks into your editor so it sweeps automatically: + +```bash +broom install +``` + +This adds (with your confirmation): + +- `~/.claude/skills/broom-sweep/SKILL.md` — a skill that teaches Claude to preview and apply redactions +- `~/.claude/hooks/stop-broom.mjs` — a Stop hook that silently scans your transcripts after every turn +- a `Stop` hook entry in `~/.claude/settings.json` to register it + +Restart Claude Code afterward for the skill to take effect. Pass `--yes` to skip the confirmation prompt. + +> **Tip:** install broomsticks globally (`npm install -g broomsticks`) when using the automatic integration. The Stop hook prefers the `broom` command but falls back to `npx broomsticks`, which re-resolves the package on every turn and adds noticeable latency to each response. + ## What it detects A curated, gitleaks-style ruleset for high-confidence provider tokens, plus an entropy-gated catch-all for unknown `KEY=value` / `"token": "…"` shapes: @@ -93,7 +168,19 @@ A curated, gitleaks-style ruleset for high-confidence provider tokens, plus an e | Connection strings | `postgres://`, `mysql://`, `mongodb+srv://`, `redis://` with inline credentials | | Generic | `api_key` / `secret` / `password` / `token` assignments above an entropy threshold | -The exact rule set, severities, and entropy thresholds live in `src/rules.mjs` once shipped — and are documented in [PLAN.md](./PLAN.md). +The exact rule set, severities, and entropy thresholds live in [`src/rules.mjs`](./src/rules.mjs). + +## Allowlist + +Well-known documentation placeholders (AWS's `AKIAIOSFODNN7EXAMPLE`, Stripe test keys, `.env.example` shapes) are suppressed out of the box. Add your own known false positives to `~/.broom/allowlist.txt`: + +``` +# one entry per line; blank lines and # comments ignored +AKIAIOSFODNN7EXAMPLE +/^sk_test_[A-Za-z0-9]+$/ +``` + +Plain lines match a secret exactly; `/regex/flags` lines match by pattern. Disable suppression entirely with `--no-allowlist`. ## How redaction works @@ -119,14 +206,18 @@ Paths shown for Linux; macOS/Windows equivalents are resolved automatically. ## Limitations -- It reduces exposure of secrets **already written to disk**; it cannot un-send anything already transmitted to a model provider. **If a secret hit a transcript, treat it as compromised and rotate it** — broomsticks is cleanup, not a substitute for rotation. +- `scan`/`clean` reduce exposure of secrets **already written to disk**; they cannot un-send anything already transmitted to a model provider. **If a secret hit a transcript before you started using the proxy, treat it as compromised and rotate it** — broomsticks is cleanup, not a substitute for rotation. - Detection is best-effort. Novel or low-entropy secrets may be missed; tune with `--extra`. - Cursor's schema evolves between versions; broomsticks targets the known chat/composer keys and will be kept current. +- The proxy buffers each streaming response in full before redacting and re-emitting it. This is deliberate — a secret straddling two SSE chunks can't be caught otherwise — but it means the assistant's UI won't show tokens incrementally; long responses appear all at once after a pause. A sliding-window buffer that preserves incremental streaming is a possible future improvement. +- The proxy scans assistant text and tool-call arguments, but **not extended-thinking blocks** — rewriting a thinking block would invalidate its cryptographic signature and break multi-turn thinking+tool loops. A secret echoed only inside a model's thinking is passed through unredacted. (Thinking is not persisted to transcripts either, so `broom clean` won't see it; if a secret reached the model at all, rotate it.) ## Contributing Issues and PRs welcome — especially new source adapters (Windsurf, Continue, Aider, Zed) and detection rules. See [PLAN.md](./PLAN.md) for architecture and the contribution surface. +Run the test suite with `npm test` (uses the built-in `node:test` runner — no dependencies). + ## License [MIT](./LICENSE) © digitaldrreamer diff --git a/bin/broom.mjs b/bin/broom.mjs index 4c64f95..50359b3 100755 --- a/bin/broom.mjs +++ b/bin/broom.mjs @@ -16,7 +16,7 @@ import { discoverTargets as claudeCodeTargets } from '../src/sources/claude-code import { discoverTargets as codexTargets } from '../src/sources/codex.mjs' import { discoverTargets as cursorTargets } from '../src/sources/cursor.mjs' import { runInstall, isInstalled } from '../src/install.mjs' -import { startProxy, installProxyEnv, installDaemon, uninstallDaemon } from '../src/proxy.mjs' +import { startProxy, installProxyEnv, uninstallProxyEnv, installDaemon, uninstallDaemon } from '../src/proxy.mjs' // ── Package metadata ────────────────────────────────────────────────────────── const pkgPath = join(dirname(fileURLToPath(import.meta.url)), '..', 'package.json') @@ -70,10 +70,18 @@ if (command === 'proxy') { if (flag('--uninstall')) { const removed = uninstallDaemon() + const cleaned = uninstallProxyEnv() console.log(removed - ? '\n broom proxy: daemon removed. Env vars in your shell profile still point to\n the proxy — remove them manually or they will silently fail to connect.\n' - : '\n broom proxy: no daemon found to remove.\n' + ? '\n broom proxy: daemon removed.' + : '\n broom proxy: no daemon found to remove.' ) + if (cleaned.length) { + console.log(' Removed proxy env vars from:') + for (const f of cleaned) console.log(` ${f}`) + console.log(' Open a new terminal (or re-source your profile) for it to take effect.\n') + } else { + console.log(' No proxy env vars found in your shell profiles.\n') + } process.exit(0) } diff --git a/src/proxy.mjs b/src/proxy.mjs index a779d9e..17d4c56 100644 --- a/src/proxy.mjs +++ b/src/proxy.mjs @@ -13,7 +13,9 @@ // Both streaming (SSE) and non-streaming responses are handled. Streaming // responses are buffered in full before redaction — a secret that straddles // two SSE chunks cannot be safely found otherwise. The client receives a -// valid synthetic SSE stream with collapsed text deltas. +// valid synthetic SSE stream with collapsed deltas. Both assistant text and +// tool-call argument JSON are scanned; extended-thinking blocks are passed +// through untouched so their signatures stay valid. import { createServer } from 'node:http' import { request as tlsReq } from 'node:https' @@ -83,26 +85,42 @@ function collect(stream) { // ── SSE redaction ───────────────────────────────────────────────────────────── // -// Strategy: accumulate the full text for each content block index across all +// Strategy: accumulate the full payload for each content block index across all // delta events, redact the complete string, then re-emit one synthetic delta -// per block with the full redacted text. All non-delta events pass through. +// per block with the full redacted payload. Both assistant text AND tool-call +// argument JSON are scanned — a model can echo a leaked secret into a tool call +// (e.g. a write_file `content` arg) just as easily as into prose. +// +// Anthropic emits each block's payload under a delta-type-specific field: +// text_delta → delta.text (assistant prose) +// input_json_delta → delta.partial_json (streamed tool-use input JSON) +// A block is exactly one type, so blockIndex → field is stable. +// +// thinking_delta / signature_delta are intentionally passed through untouched: +// extended-thinking blocks are cryptographically signed, and rewriting the text +// would invalidate the signature and break multi-turn thinking+tool loops. +const ANTHROPIC_DELTA_FIELD = { text_delta: 'text', input_json_delta: 'partial_json' } -function redactAnthropicSSE(lines, allowlist, vault) { - const accumulated = new Map() // blockIndex → full text +export function redactAnthropicSSE(lines, allowlist, vault) { + const acc = new Map() // blockIndex → { field, content } for (const line of lines) { if (!line.startsWith('data:')) continue const raw = line.slice(5).trim() if (!raw || raw === '[DONE]') continue let ev; try { ev = JSON.parse(raw) } catch { continue } if (!ev || typeof ev !== 'object') continue - if (ev.type === 'content_block_delta' && ev.delta?.type === 'text_delta') { + if (ev.type === 'content_block_delta') { + const field = ANTHROPIC_DELTA_FIELD[ev.delta?.type] + if (!field) continue const i = ev.index ?? 0 - accumulated.set(i, (accumulated.get(i) ?? '') + (ev.delta.text ?? '')) + const prev = acc.get(i) ?? { field, content: '' } + prev.content += ev.delta[field] ?? '' + acc.set(i, prev) } } const redacted = new Map() - for (const [i, text] of accumulated) redacted.set(i, redactStr(text, allowlist, vault)) + for (const [i, { content }] of acc) redacted.set(i, redactStr(content, allowlist, vault)) const emitted = new Set() const out = [] @@ -112,15 +130,16 @@ function redactAnthropicSSE(lines, allowlist, vault) { if (!raw || raw === '[DONE]') { out.push(line); continue } let ev; try { ev = JSON.parse(raw) } catch { out.push(line); continue } - if (ev.type === 'content_block_delta' && ev.delta?.type === 'text_delta') { + const field = ev.type === 'content_block_delta' && ANTHROPIC_DELTA_FIELD[ev.delta?.type] + if (field && acc.has(ev.index ?? 0)) { const i = ev.index ?? 0 if (!emitted.has(i)) { emitted.add(i) out.push('data: ' + JSON.stringify({ - ...ev, delta: { ...ev.delta, text: redacted.get(i) ?? '' }, + ...ev, delta: { ...ev.delta, [field]: redacted.get(i) ?? '' }, })) } - // drop subsequent deltas — text already emitted above + // drop subsequent deltas — full payload already emitted above } else { out.push(line) } @@ -128,8 +147,14 @@ function redactAnthropicSSE(lines, allowlist, vault) { return out } -function redactOpenAISSE(lines, allowlist, vault) { - const accumulated = new Map() // choiceIndex → full content string +// OpenAI streams assistant text under choices[].delta.content and tool-call +// input under choices[].delta.tool_calls[].function.arguments (fragmented, +// keyed by the tool_call's own `index`). Both carry secrets a model may echo, +// so both are accumulated, redacted, and collapsed into their first occurrence. +export function redactOpenAISSE(lines, allowlist, vault) { + const accText = new Map() // choiceIndex → content string + const accArgs = new Map() // `${choiceIndex}:${toolIndex}` → arguments string + for (const line of lines) { if (!line.startsWith('data:')) continue const raw = line.slice(5).trim() @@ -137,17 +162,26 @@ function redactOpenAISSE(lines, allowlist, vault) { let ev; try { ev = JSON.parse(raw) } catch { continue } if (!ev || typeof ev !== 'object') continue for (const c of ev.choices ?? []) { + const ci = c.index ?? 0 if (typeof c.delta?.content === 'string') { - const i = c.index ?? 0 - accumulated.set(i, (accumulated.get(i) ?? '') + c.delta.content) + accText.set(ci, (accText.get(ci) ?? '') + c.delta.content) + } + for (const tc of c.delta?.tool_calls ?? []) { + if (typeof tc.function?.arguments === 'string') { + const key = `${ci}:${tc.index ?? 0}` + accArgs.set(key, (accArgs.get(key) ?? '') + tc.function.arguments) + } } } } - const redacted = new Map() - for (const [i, text] of accumulated) redacted.set(i, redactStr(text, allowlist, vault)) + const redText = new Map() + for (const [k, v] of accText) redText.set(k, redactStr(v, allowlist, vault)) + const redArgs = new Map() + for (const [k, v] of accArgs) redArgs.set(k, redactStr(v, allowlist, vault)) - const emitted = new Set() + const emittedText = new Set() + const emittedArgs = new Set() const out = [] for (const line of lines) { if (!line.startsWith('data:')) { out.push(line); continue } @@ -155,15 +189,32 @@ function redactOpenAISSE(lines, allowlist, vault) { if (!raw || raw === '[DONE]') { out.push(line); continue } let ev; try { ev = JSON.parse(raw) } catch { out.push(line); continue } - const hasContent = (ev.choices ?? []).some(c => typeof c.delta?.content === 'string') - if (!hasContent) { out.push(line); continue } + const hasText = (ev.choices ?? []).some(c => typeof c.delta?.content === 'string') + const hasArgs = (ev.choices ?? []).some(c => + (c.delta?.tool_calls ?? []).some(tc => typeof tc.function?.arguments === 'string')) + if (!hasText && !hasArgs) { out.push(line); continue } const newChoices = (ev.choices ?? []).map(c => { - if (typeof c.delta?.content !== 'string') return c - const i = c.index ?? 0 - if (emitted.has(i)) return { ...c, delta: { ...c.delta, content: '' } } - emitted.add(i) - return { ...c, delta: { ...c.delta, content: redacted.get(i) ?? '' } } + const ci = c.index ?? 0 + let delta = c.delta + + if (typeof delta?.content === 'string') { + if (emittedText.has(ci)) delta = { ...delta, content: '' } + else { emittedText.add(ci); delta = { ...delta, content: redText.get(ci) ?? '' } } + } + + if (Array.isArray(delta?.tool_calls)) { + const tcs = delta.tool_calls.map(tc => { + if (typeof tc.function?.arguments !== 'string') return tc + const key = `${ci}:${tc.index ?? 0}` + if (emittedArgs.has(key)) return { ...tc, function: { ...tc.function, arguments: '' } } + emittedArgs.add(key) + return { ...tc, function: { ...tc.function, arguments: redArgs.get(key) ?? '' } } + }) + delta = { ...delta, tool_calls: tcs } + } + + return { ...c, delta } }) out.push('data: ' + JSON.stringify({ ...ev, choices: newChoices })) } @@ -296,6 +347,41 @@ export function installProxyEnv(port = 7777) { return updated } +// Matches the whole block installProxyEnv appends: the marker line, the body, +// and the closing rule line. The opening line has only two consecutive box +// chars ("# ── broomsticks…"), so `# ─{3,}` reliably anchors to the closing +// rule and never to the opener. A leading newline (added when the block is +// appended) is consumed so uninstall restores the original file shape. +const PROXY_BLOCK_RE = /\n?# ── broomsticks proxy[\s\S]*?\n# ─{3,}[^\n]*\n?/g + +/** + * Remove the proxy env-var block that installProxyEnv added. Inverse of + * installProxyEnv: strips the marked block from every shell init file that + * contains it, leaving the rest of the file untouched. + * + * @returns {string[]} paths that were updated + */ +export function uninstallProxyEnv() { + const home = homedir() + const updated = [] + + for (const name of PROFILE_CANDIDATES) { + const file = join(home, name) + if (!existsSync(file)) continue + try { + const contents = readFileSync(file, 'utf8') + if (!contents.includes(BROOM_MARKER)) continue // nothing to remove + const stripped = contents.replace(PROXY_BLOCK_RE, '') + if (stripped !== contents) { + writeFileSync(file, stripped, 'utf8') + updated.push(file) + } + } catch { /* permission error or race — skip silently */ } + } + + return updated +} + // ── Daemon installation ─────────────────────────────────────────────────────── function launchdPlist(nodeBin, broomBin, port) { diff --git a/test/proxy-env.test.mjs b/test/proxy-env.test.mjs new file mode 100644 index 0000000..2eb8e19 --- /dev/null +++ b/test/proxy-env.test.mjs @@ -0,0 +1,51 @@ +// Proxy shell-profile env install/uninstall round-trip. installProxyEnv appends +// a marked block; uninstallProxyEnv must remove exactly that block and leave +// the user's own lines untouched. + +import { test } from 'node:test' +import assert from 'node:assert/strict' +import { mkdtempSync, writeFileSync, readFileSync, rmSync } from 'node:fs' +import { tmpdir, homedir } from 'node:os' +import { join } from 'node:path' + +import { installProxyEnv, uninstallProxyEnv } from '../src/proxy.mjs' + +test('installProxyEnv → uninstallProxyEnv restores the original profile', (t) => { + const home = mkdtempSync(join(tmpdir(), 'broom-env-')) + const prevHome = process.env.HOME + process.env.HOME = home + + try { + // Only meaningful if os.homedir() honors our $HOME override on this platform. + if (homedir() !== home) { + return t.skip('homedir() not redirectable via $HOME on this platform') + } + + const rc = join(home, '.zshrc') + const original = 'export PATH="$HOME/bin:$PATH"\nalias ll="ls -la"\n' + writeFileSync(rc, original, 'utf8') + + // Install: block appended, marker present, user content preserved. + const added = installProxyEnv(7777) + assert.deepEqual(added, [rc], 'install should report the .zshrc it touched') + const afterInstall = readFileSync(rc, 'utf8') + assert.ok(afterInstall.includes('ANTHROPIC_BASE_URL=http://127.0.0.1:7777')) + assert.ok(afterInstall.includes('OPENAI_BASE_URL=http://127.0.0.1:7777')) + assert.ok(afterInstall.startsWith(original), 'user content stays at the top') + + // Uninstall: block gone, user content byte-for-byte intact. + const removed = uninstallProxyEnv() + assert.deepEqual(removed, [rc], 'uninstall should report the .zshrc it cleaned') + const afterUninstall = readFileSync(rc, 'utf8') + assert.ok(!afterUninstall.includes('broomsticks proxy'), 'marker removed') + assert.ok(!afterUninstall.includes('ANTHROPIC_BASE_URL'), 'env var removed') + assert.equal(afterUninstall, original, 'profile restored to its original bytes') + + // Idempotent: uninstall again is a no-op. + assert.deepEqual(uninstallProxyEnv(), [], 'second uninstall touches nothing') + } finally { + if (prevHome === undefined) delete process.env.HOME + else process.env.HOME = prevHome + rmSync(home, { recursive: true, force: true }) + } +}) diff --git a/test/proxy-sse.test.mjs b/test/proxy-sse.test.mjs new file mode 100644 index 0000000..4df4a56 --- /dev/null +++ b/test/proxy-sse.test.mjs @@ -0,0 +1,115 @@ +// SSE redaction — regression guard for the tool-call leak: a secret echoed by +// the model inside a tool-call argument (split across streamed delta chunks) +// must be redacted before the client sees it, and the event structure must +// survive so the client can still reconstruct the tool call. + +import { test } from 'node:test' +import assert from 'node:assert/strict' + +import { redactAnthropicSSE, redactOpenAISSE } from '../src/proxy.mjs' + +// Assembled from parts so the literal token never appears in-file; shaped to +// match the github-pat rule (ghp_ + 36 alphanumerics). +const SECRET = 'ghp_' + '1234567890abcdefABCDEF1234567890abcd' + +/** Join emitted `data:` JSON events back into one string for assertions. */ +function joined(lines) { + return lines.join('\n') +} + +test('anthropic: secret split across input_json_delta chunks is redacted', () => { + const vault = new Map() + const half = SECRET.length >> 1 + const lines = [ + 'event: content_block_start', + 'data: ' + JSON.stringify({ type: 'content_block_start', index: 0, + content_block: { type: 'tool_use', id: 'toolu_1', name: 'write_file', input: {} } }), + '', + 'event: content_block_delta', + 'data: ' + JSON.stringify({ type: 'content_block_delta', index: 0, + delta: { type: 'input_json_delta', partial_json: '{"content": "' + SECRET.slice(0, half) } }), + '', + 'event: content_block_delta', + 'data: ' + JSON.stringify({ type: 'content_block_delta', index: 0, + delta: { type: 'input_json_delta', partial_json: SECRET.slice(half) + '"}' } }), + '', + 'event: content_block_stop', + 'data: ' + JSON.stringify({ type: 'content_block_stop', index: 0 }), + ] + + const out = redactAnthropicSSE(lines, null, vault) + const text = joined(out) + + assert.ok(!text.includes(SECRET), 'reassembled tool-call args must not contain the secret') + assert.ok(text.includes('write_file') && text.includes('toolu_1'), 'tool-call structure preserved') + assert.ok(vault.size === 1, 'secret captured in the vault') + + // The collapsed partial_json fragments must still reassemble into valid JSON. + const partials = out + .filter(l => l.startsWith('data:')) + .map(l => { try { return JSON.parse(l.slice(5)) } catch { return null } }) + .filter(ev => ev?.delta?.type === 'input_json_delta') + .map(ev => ev.delta.partial_json) + .join('') + const parsed = JSON.parse(partials) + assert.ok(parsed.content.startsWith('«BROOM:'), 'secret replaced with a placeholder') +}) + +test('anthropic: text_delta path still redacts (no regression)', () => { + const vault = new Map() + const lines = [ + 'data: ' + JSON.stringify({ type: 'content_block_delta', index: 0, + delta: { type: 'text_delta', text: 'here is the key ' + SECRET } }), + ] + const out = redactAnthropicSSE(lines, null, vault) + assert.ok(!joined(out).includes(SECRET)) + assert.equal(vault.size, 1) +}) + +test('anthropic: thinking_delta is passed through unmodified (signature safety)', () => { + const vault = new Map() + const original = 'data: ' + JSON.stringify({ type: 'content_block_delta', index: 0, + delta: { type: 'thinking_delta', thinking: 'let me reason about this' } }) + const out = redactAnthropicSSE([original], null, vault) + assert.deepEqual(out, [original], 'thinking deltas must pass through byte-for-byte') +}) + +test('openai: secret split across tool_calls arguments chunks is redacted', () => { + const vault = new Map() + const half = SECRET.length >> 1 + const lines = [ + 'data: ' + JSON.stringify({ choices: [{ index: 0, delta: { tool_calls: [ + { index: 0, id: 'call_1', type: 'function', + function: { name: 'write_file', arguments: '{"content":"' + SECRET.slice(0, half) } } ] } }] }), + 'data: ' + JSON.stringify({ choices: [{ index: 0, delta: { tool_calls: [ + { index: 0, function: { arguments: SECRET.slice(half) + '"}' } } ] } }] }), + 'data: [DONE]', + ] + + const out = redactOpenAISSE(lines, null, vault) + const text = joined(out) + + assert.ok(!text.includes(SECRET), 'reassembled tool-call args must not contain the secret') + assert.ok(text.includes('write_file') && text.includes('call_1'), 'tool-call structure preserved') + assert.equal(vault.size, 1) + + const args = out + .filter(l => l.startsWith('data:') && l.slice(5).trim() !== '[DONE]') + .flatMap(l => { try { return JSON.parse(l.slice(5)).choices ?? [] } catch { return [] } }) + .flatMap(c => c.delta?.tool_calls ?? []) + .map(tc => tc.function?.arguments ?? '') + .join('') + const parsed = JSON.parse(args) + assert.ok(parsed.content.startsWith('«BROOM:'), 'secret replaced with a placeholder') +}) + +test('openai: content path still redacts (no regression)', () => { + const vault = new Map() + const lines = [ + 'data: ' + JSON.stringify({ choices: [{ index: 0, delta: { content: 'key: ' + SECRET } }] }), + 'data: [DONE]', + ] + const out = redactOpenAISSE(lines, null, vault) + assert.ok(!joined(out).includes(SECRET)) + assert.equal(vault.size, 1) +})