Skip to content

Keep conversation content out of logs and refuse third-party tracing - #227

Open
mfafore22 wants to merge 2 commits into
mainfrom
fix/no-content-in-logs
Open

mfafore22 wants to merge 2 commits into
mainfrom
fix/no-content-in-logs

Conversation

@mfafore22

Copy link
Copy Markdown
Member

Closes #191

ADR 0011 says no observability tool may see conversation content. Three things got in the way of that.

LangChain's tracing client is installed, and setting LANGSMITH_TRACING=true would send every prompt to LangSmith. The backend now refuses to start if any LangSmith/LangChain tracing flag is on, or if a tracing API key or endpoint is set. The error names the variable but never prints its value.

Exception messages are where user text usually leaks: a provider error quoting the prompt, a validation error quoting the rejected value. A log record factory now rewrites every logged exception to its type and location (file, line, function, and the chain it was raised from) and drops the message. It works for every logger and handler, so it also covers dependencies. I also fixed the two explicit log calls that broke the rule: the guardrail line logged the model's reason, which can quote the message, and the Ollama connection warning logged the raw exception.

While writing the canary test I found a third leak: at DEBUG level, sse_starlette logs every chunk it streams (every model reply), and the openai and anthropic clients log full request bodies. Those three loggers are now held at INFO even when the app runs at DEBUG.

There's no error-reporting integration yet, so there's nothing to scrub. When #38 adds one, it needs a before_send that drops request and response bodies.

The issue also asks for a test through admission and sign-in. Those don't exist yet, so the test covers the chat flow. It sends a canary string through a normal turn, a guardrailed turn and a failing turn, captures every logger at DEBUG, and checks the canary appears nowhere. It should be extended when #184 and #185 land.

328 tests pass; the two test_prompt_loader failures already fail on main (Windows file locking). Ruff and mypy are clean.

@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 220e23cf-f3fd-4f0c-a944-86252849ef5b
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

CI Results

Service Check Result
crawler ruff (lint) ✅ Passed
crawler mypy ✅ Passed
crawler pyrefly ✅ Passed
crawler pytest + coverage ✅ Passed
ingest ruff (lint) ✅ Passed
ingest mypy ✅ Passed
ingest pyrefly ✅ Passed
ingest pytest + coverage ✅ Passed
register ruff (lint) ✅ Passed
register mypy ✅ Passed
register pyrefly ✅ Passed
register pytest + coverage ✅ Passed
backend ruff (lint) ✅ Passed
backend mypy ✅ Passed
backend pyrefly ✅ Passed
backend pytest + coverage ✅ Passed
app lint (eslint + prettier) ✅ Passed

All checks passed. 🎉

Comment thread backend/app/log_privacy.py Resolved
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Keep conversation content out of logs, traces, and error reports

1 participant