feat(sast): detect bearer DNS validation TOCTOU

.github/workflows/tests.yml

@@ -16,7 +16,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ['3.11', '3.13']
+        python-version: ['3.11', '3.12', '3.13']
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:

CHANGELOG.d/892-bearer-dns-toctou-detector.md

@@ -0,0 +1,6 @@
+## Security
+
+- Add the HIGH built-in Bearer DNS-rebinding TOCTOU detector family for Python `urllib` flows that validate a URL before dispatch but allow the network client to make a second DNS decision. The source-backed regression corpus preserves the pre-PR #898 vulnerable flow and the protected DNS-pinned HTTPS repair for security issue #892.
+- Track post-construction request state instead of treating the original constructor as permanently authoritative: opaque/dynamic Authorization replacement, header mapping replacement/clearing, and unrelated `Request.full_url` mutation terminate stale credential or destination provenance when those operations affect the active credential/destination. A narrowly scoped companion rule keeps provably Bearer-valued variable replacements detectable, while self-derived destination mutations remain positive. All companion rules bind the validated endpoint only when it is the actual first positional Request URL or `url=...`, not when it merely appears in another argument.
+- Preserve pre-Request destination and bounded control-flow provenance: unrelated reassignment of the validated URL or its derived endpoint breaks the dynamic-Bearer path, while self-derived transformations remain detectable; a Bearer mutation under `if flag:` cannot donate credential state to a resolver-backed dispatch nested under a later `if not flag:`, while direct fallthrough from that guarded mutation to an outer dispatch remains positive.
+- Model urllib credential storage and precedence precisely for `add_unredirected_header`: a Bearer value in `unredirected_hdrs` survives ordinary `req.headers.clear()`, empty-map replacement, or Authorization pop, but a same-name normal Authorization entry overrides it in `Request.header_items()` at dispatch. `remove_header("Authorization")` removes both stores. Production scanner regressions and runtime `urllib.request.Request` assertions preserve clear/pop re-exposure, normal-header override, direct/multiline positives, destination-binding negatives, and the universal-removal negative.

docs/TRACEABILITY.md

@@ -1,7 +1,7 @@
 # AppGuardrail Requirements, Detection, and Evidence Traceability
 
 **Status:** Accepted cross-cutting baseline  
-**Last reviewed:** 2026-08-12
+**Last reviewed:** 2026-09-02
 
 | Requirement / security class | Detector/control boundary | Evidence maturity |
 |---|---|---|
@@ -19,12 +19,14 @@
 | every retained issue claim mapped to executable detector obligation | issue-detection audit | PR #911 active-PR |
 | authenticated workflow-result detector evidence | issue-detection audit workflow evidence | PR #911 active-PR |
 | automatic scanner detection of unsafe stored-webhook SSRF pattern | built-in `python-stored-ssrf-webhook-url` rule | implemented-main through PR #910 for tested Python `set_webhook` direct and one-hop persistence flows; bounded scope |
+| bearer-authenticated DNS rebinding prevention | DNS-pinned HTTPS control-plane transport | implemented-main through PR #898 for the reviewed scan-delivery boundary |
+| automatic scanner detection of preflight DNS-validation TOCTOU before bearer urllib dispatch | built-in `python-bearer-preflight-dns-toctou`, `python-bearer-preflight-dns-toctou-header-mutation`, `python-bearer-preflight-dns-toctou-multiline-constructor`, `python-bearer-preflight-dns-toctou-multiline-header-mutation`, `python-bearer-preflight-dns-toctou-dynamic-bearer-replacement`, and `python-bearer-preflight-dns-toctou-unredirected-header-persistence` detector family | integration evidence: PR #1080; source-backed vulnerable/fixed fixtures and production `_scan_file` regressions must remain with the detector family |
 | structural Semgrep-style `pattern:` execution by lightweight engine | built-in scanner | not implemented unless a real structural matcher is added; fixtures are not execution |
 
 ## Promotion rules
 
 - `implemented-main` requires source/tests on protected `develop`, not an issue/PR description.
-- `active-PR` becomes current only after merge plus fresh protected-head required evidence.
+- A PR reference records candidate/integration evidence only; `implemented-main` becomes current only after merge plus fresh protected-head required evidence.
 - External-engine capability must name the engine and availability; normalization does not convert it into a built-in detector.
 - A prevention/hardening change does not automatically promote the matching scanner-detection row; PR #924 and PR #910 were verified and promoted independently.
 - An issue registry mapping cannot promote an obligation unless actual detector execution derives its result from independent/closed evidence.
@@ -47,10 +49,28 @@ For stored webhook/callback SSRF, trace separately:
 
 Current protected-branch evidence keeps those controls distinct: PR #924 supplies the fail-closed webhook storage boundary, and PR #910 supplies the packaged `python-stored-ssrf-webhook-url` detector plus focused regression corpus. Neither control expands the detector beyond its declared source/sink and flow contract.
 
+## Bearer DNS TOCTOU traceability
+
+Security issue #892 and merged PR #898 establish the runtime defect and repair: a bearer-authenticated control-plane push first validated a URL, then `urllib` made a second DNS decision during connection; the protected repair uses DNS-pinned HTTPS so the actual connection uses the validated public address set while retaining the hostname for TLS identity verification.
+
+PR #1080 is the integration record for the independent scanner obligation. The detector family keeps six executable rule identities rather than overloading one rule ID or forcing materially different syntax/state forms through one expression. `python-bearer-preflight-dns-toctou` binds the same-function source-derived path `_is_safe_url(url)` preflight -> endpoint derived from that URL -> executable `urllib.request.Request` whose direct `headers=` argument carries an `Authorization: Bearer ...` credential -> later reviewed re-resolving urllib dispatch. `python-bearer-preflight-dns-toctou-header-mutation` covers the same validated-destination race when Bearer authorization is added or restored on the still-live tracked request after construction. `python-bearer-preflight-dns-toctou-multiline-constructor` and `python-bearer-preflight-dns-toctou-multiline-header-mutation` cover the corresponding reviewed line-wrapped credential forms without changing the defect identity or broadening the transport boundary. `python-bearer-preflight-dns-toctou-dynamic-bearer-replacement` preserves detection when a local replacement variable is provably Bearer-valued before being applied to the tracked request. `python-bearer-preflight-dns-toctou-unredirected-header-persistence` models urllib's separate `unredirected_hdrs` credential store and the normal-header precedence used by `Request.header_items()`: clearing/emptying/popping the ordinary Authorization entry can expose a still-live unredirected Bearer value, while a same-name normal Authorization entry overrides that unredirected value at dispatch. The historical vulnerable fixture and the reviewed pinned-HTTPS fixed fixture remain regression oracles. Production `_scan_file` negatives preserve the reviewed false-positive boundary: commented-out request/header/dispatch text cannot donate evidence; request construction and dispatch in mutually exclusive branches cannot form one path; a Bearer mutation under a simple `if flag:` cannot donate credential state to a resolver-backed dispatch nested under a later `if not flag:`; unauthenticated urllib delivery, validation without network dispatch, arbitrary custom/pinned opener transports, and evidence split across sibling functions are not this detector family. The inverse-guard veto is deliberately narrow: the same guarded mutation followed directly by an outer unconditional resolver-backed dispatch remains positive. Direct `urllib.request.urlopen` and the reviewed `urllib.request.build_opener(SafeRedirectHandler()).open` flow remain positive network sinks because they can repeat hostname resolution after the separate preflight.
+
+The multiline companion rules inherit the same destination, request-identity, credential-state, and reachability barriers as their direct-layout family members. Before Request construction, replacing the validated URL or URL-derived endpoint with an unrelated destination breaks destination provenance. After Request construction, reassigning the endpoint variable alone does not sanitize the already-bound Request; before dispatch, tracked-request replacement, unconditional same-path `return`/`raise`, Authorization removal, or a statically non-Bearer Authorization overwrite terminates the corresponding live request path when that operation actually targets the credential store that carries or overrides Authorization. Self-derived endpoint/request state remains eligible before it is bound, and a later supported Bearer restoration can re-establish credential provenance. `tests/test_bearer_dns_toctou_multiline_regressions.py` executes these boundaries through production `_scan_file`, including paired vulnerable, sanitized, dead-sink, self-derived, fully multiline constructor-plus-mutation, and remove-then-restore cases. `tests/test_bearer_dns_toctou_inverse_condition_regression.py` keeps one-line and multiline inverse-condition negatives paired with outer-fallthrough positives so control-flow precision cannot be recovered by silently dropping executable vulnerable paths.
+
+Request construction is destination- and credential-bound rather than token-text-bound. Every family member, including the dynamic-Bearer and unredirected-header companions, requires the tracked endpoint to be the actual `Request` destination through the first positional URL argument or `url=endpoint`; an endpoint mentioned only in another header/argument beside a fixed request URL cannot donate destination provenance. For the primary Request-credential path, `Authorization: Bearer ...` counts only when it is in the supported direct `headers=` argument at Request-call argument indentation; a nested `headers=` expression inside `data=` or another argument is not HTTP-header evidence. Ordinary direct Request keyword arguments such as `data=` and `method=` may appear between the tracked URL argument and the direct `headers=` argument without breaking the path. Replacing the validated URL with an unrelated destination before endpoint derivation breaks preflight provenance, while self-derived transformations of the validated URL preserve it.
+
+Credential-removal and request-replacement barriers are path-sensitive for the supported nested layouts. A same-branch `remove_header("Authorization")` before dispatch is a universal credential barrier for urllib Request header storage unless a supported later Bearer mutation re-establishes credential flow. For a credential stored in the ordinary `req.headers` mapping, `headers.clear()`, `headers = {}`, `headers.pop("Authorization", ...)`, and a non-Bearer same-name overwrite terminate Bearer provenance. For a Bearer value installed through `add_unredirected_header`, the storage and transmission semantics differ: ordinary-map clear/empty/pop operations do not delete `unredirected_hdrs` and can leave or reveal that Bearer value, whereas a normal same-name Authorization entry takes precedence in `Request.header_items()` and therefore suppresses transmission of the unredirected Bearer until that overriding normal entry is removed. `remove_header("Authorization")` removes both stores. Removal confined to a different branch does not sanitize a dispatch on the branch where the credential remains. Likewise, a same-branch replacement of the tracked request breaks the original path whenever the replacement destination is not the tracked endpoint, regardless of whether the replacement itself carries Bearer credentials. A rebuild that still targets the tracked endpoint remains detectable when credential provenance also remains, while an unauthenticated same-endpoint rebuild is a credential break. An opposite-branch replacement cannot sanitize a sibling branch that still dispatches the original Bearer request. The production review-boundary regressions preserve these paired cases together with first-positional and keyword-URL positives so later regex changes cannot recover precision by creating silent false negatives.
+
+Post-construction Bearer mutation is an independent credential source for the header-mutation subrule and does not require a preceding removal. `add_header("Authorization", "Bearer ...")`, `add_unredirected_header(...)`, or `headers["Authorization"] = "Bearer ..."` can therefore establish the credential-bearing DNS-re-resolution path on an initially unauthenticated tracked Request; the same forms also restore the path after an explicit removal. The unredirected-header persistence companion is intentionally narrower: it preserves a finding after ordinary-header clear, empty-map replacement, or Authorization pop because those operations leave the unredirected store live, but it stops when a normal same-name Authorization value overrides that store and treats `remove_header("Authorization")` as the supported operation that clears both stores. `tests/test_bearer_dns_toctou_unredirected_persistence.py` fixes that contract through production `_scan_file` and runtime `urllib.request.Request.header_items()` assertions, including normal-header override, pop/clear re-exposure, direct and multiline credential installation, destination-binding negatives, and a `remove_header` negative. The mutation path still requires the validated destination and request identity to remain live through the reviewed re-resolving sink, and it breaks provenance when the validated URL/endpoint is replaced before Request construction, the tracked request is replaced before dispatch, a non-Bearer authorization actually replaces the active credential, or an unconditional same-path `return`/`raise` makes the sink unreachable. Self-derived endpoint updates and request-preserving assignments remain positive so false-positive barriers do not become silent false-negative sanitizers. Bearer-looking text nested in request data is not used as HTTP-header evidence.
+
+Simple self-derived assignments preserve the tracked path when an executable tracked identifier appears before any string literal or comment on the right-hand side, including `endpoint = endpoint + ...`, URL-derived endpoint updates, and `req = req`. An assignment where the tracked name occurs only as quoted data, such as `endpoint = choose("endpoint")` or `req = choose("req")`, is a provenance break and must not create a HIGH finding. More general wrapper/helper-mediated value flow that places a literal before the tracked identifier is outside this bounded regex detector and requires separate executable evidence rather than speculative flow inference. Paired production `_scan_file` regressions keep both the self-derived positives and quoted-name replacement negatives stable.
+
+The detector deliberately does not claim general interprocedural or cross-library DNS-rebinding analysis. Alternative HTTP clients, helper/cross-file flows, differently named validation boundaries, richer credential construction expressions, arbitrary Request argument ordering beyond the supported direct-header layouts, and custom transports whose actual socket connection is independently pinned require separate evidence. Do not infer scanner maturity from the already-merged runtime repair; promotion requires the detector and its regression corpus on protected `develop` plus fresh protected-head required evidence.
+
 ## Standards/research
 
 Existing repository docs/doctoring/security evidence remain the bibliography/source-of-truth for standards such as SARIF, CycloneDX, GitHub security interfaces, and applicable OWASP/CWE classes. Material new detector classes should add authoritative standard/CWE/OWASP references and APA 7 citations in doctoring where research/standards materially drive implementation.
 
 ## Change rule
 
-Every new issue-class detector or product security boundary should add/update a row and its concrete test/evidence path. Stale/queued/cancelled/rate-limited/predecessor checks cannot promote evidence maturity.
+Every new issue-class detector or product security boundary should add/update a row and its concrete test/evidence path. Stale/queued/cancelled/rate-limited/predecessor checks cannot promote evidence maturity.