Skip to content

fix(scan): audit graph draws immediate dependencies, not the soldeer closure - #177

Merged
thedavidmeister merged 3 commits into
masterfrom
audit-graph-immediate-deps
Aug 19, 2026
Merged

fix(scan): audit graph draws immediate dependencies, not the soldeer closure#177
thedavidmeister merged 3 commits into
masterfrom
audit-graph-immediate-deps

Conversation

@thedavidmeister

@thedavidmeister thedavidmeister commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

The defect

graph_edges() drew one edge per [dependencies] entry. Soldeer does not
resolve transitive dependencies, so an org manifest declares the FULL closure —
a consumer must declare every package its dependencies' own internal
rain-solmem-0.1.3/... imports resolve through, whether or not its own code
ever imports them. The audit graph therefore drew transitive edges as if they
were direct: rain.verify showed 16 first-party outgoing edges when its own code
imports 6 first-party packages; edge count tracked manifest size, not usage.

The fix — immediacy from data, never from graph shape

An edge from → to survives iff a file in from's OWN tree imports to's
package prefix.

  • Own tree is the shared not-vendored property (untested::is_vendored):
    everything outside top-level lib/, dependencies/, node_modules/, out/
    and cache/src/, script/ and test/ in the org's layout, without
    dropping own code that lives elsewhere. A vendored file's imports are its own
    package's dependencies resolving through the consumer's remappings — counting
    them would re-admit the exact closure this cuts.
  • Imports are AST-parsed (solang_parser::parse), so a commented-out
    import or an import-shaped string literal is not a use. A file the grammar
    rejects (solang can lag solc syntax) still lexes; a token-stream fallback
    reads import … one string literal … ; rather than silently dropping a
    real direct edge.
  • The prefix matches a package by exact name or <package>- + a
    leading-digit version — never hyphen-splitting, so rain-math-float-0.0.1
    is not a use of rain-math, while pre-release
    rain-math-float-0.0.1-alpha.5 is one of rain-math-float. Deliberately
    version-agnostic: an import pinning an older path than the manifest is still
    a direct use, and version/stale data on surviving edges keeps coming from
    the manifest exactly as before (Dependency graph ignores pinned versions — a stale pin's transitive closure can differ from what's drawn #79 semantics untouched).
  • This is NOT transitive reduction. A package imported directly AND
    reachable through another dependency keeps its edge (everything imports
    solmem); a test asserts precisely that. Immediacy is decided from the imports
    that exist, never from the shape of the graph.
  • Unknown stays typed and loud. Declared dependencies + unreadable tree =
    the same deps_known: false state an unparseable manifest yields (typed
    DepsResolution, ::warning in the scan log), never a silent keep-all or
    drop-all. A manifest declaring nothing needs no tree to confirm it. The site
    label now says which ("deps unknown — manifest or tree unreadable") — the
    only site change; the data shape is unchanged.
  • Scan cost: zero additional requests. The prefixes are collected from the
    same shallow-clone walk untested::analyze already does. No coverage bound
    was added anywhere; nothing is skipped silently.

Verification: two full 4-org scans, same token, same minute

The hourly cron's committed site/health.json is NOT a fair baseline here: it
runs with a token that sees only public repos, while a local scan sees the
private ones too (st0x.atomic-bridge, st0x.bebop, st0x.univ4.hook, dvin.poc
have no node in it at all). So the baseline below is master's own binary
built and run locally
(same token, same ORGS="rainlanguage cyclofinance S01-Issuer ST0x-Technology"), 11 minutes apart from the branch scan.

  • Node sets: identical, 47 Solidity repos.

  • Edges: 190 → 115 (−75). Strict subset: zero edges in AFTER that are
    not in BEFORE.

  • 33 of 47 repos unchanged — a repo already declaring only what it imports
    (e.g. rain.deploy) is untouched. The 14 that changed, all decreases:

    repo before after
    rain.verify 16 5
    raindex 14 9
    rain.dia 14 8
    rain.erc4626.words 14 5
    rain.flare 14 6
    rainlang 14 12
    rain.merkle 14 4
    rain.pyth 14 6
    st0x.deploy 10 8
    rain.vats.flare 8 3
    st0x.atomic-bridge 7 1
    st0x.oracle 5 4
    rain.extrospection.deploy 3 2
    rain.tofu.erc20-decimals 3 2
  • Ground truth, checked against fresh clones rather than the scan's own output:

    • rain.verify owns imports of exactly 6 first-party packages
      (rain-factory, rain-verify-interface, rain-deploy,
      rain-interpreter-interface, rain-solmem, rain-sol-codegen). Five map
      to graph nodes = the 5 kept edges. The sixth, rain-interpreter-interface,
      is the LEGACY name of today's rainlang-interface (repo
      rainlang.interface) — no node publishes the old name, so it drew no edge on
      master either. Pre-existing, unchanged by this PR, noted for follow-up.
    • rain.merkle imports 4 node-mapped packages → exactly its 4 kept edges;
      rain.erc4626.words imports 5 → exactly its 5. st0x.atomic-bridge
      declares 7 first-party packages and imports exactly one
      (rain-math-float-0.1.1) → its 1 kept edge; the other 6 are entries no
      import justifies.

blockedBy — identical for 45 of 47 nodes, and the two diffs are the fix working

Reachability over direct edges equals reachability over closure edges whenever
every closure entry really sits beneath some direct dependency — a unit test
asserts blocker sets are unchanged in that case. The two live diffs are repos
whose manifests carry dead entries (declared, imported nowhere, and beneath
no kept dependency either):

  • rain.verify 12 → 4. It declares rainlang = "0.1.2" and imports it
    nowhere. The 8 lost blockers are exactly rainlang plus rainlang's own
    unaudited subtree (rain.datacontract, rain.extrospection,
    rain.lib.memkv, rain.math.float, rain.metadata, rain.string,
    rain.tofu.erc20-decimals) minus what stays reachable through kept edges
    (rain.deploy, rain.sol.codegen, rain.solmem all remain).
  • st0x.atomic-bridge 15 → 6. Six of its seven declared first-party entries
    are imported nowhere in its own code. What remains is precisely
    rain.math.float + rain.math.float's subtree (rain.datacontract,
    rain.deploy, rain.sol.codegen, rain.solmem, rain.string) —
    rain.math.float's manifest and imports confirm that subtree exactly.

Master asserted these repos stand on ground their code does not touch. The
manifests still carry the dead entries (the scan is not a linter for them);
the graph just stops drawing them as dependencies.

Before / after renders

Same page (site/audit.html), same scan universe, only the binary differs.
The closure thicket collapses to the direct graph; the headline "11 of 47 …
clear ground" is unchanged; most-inherited moves only where the blockedBy
shrinks predict (rain.factory 7 → 6 inherit; rain.solmem 28 and
rain.sol.codegen 25 unchanged); per-repo stale-deps lists shrink in step with
their dropped edges (rain.merkle 8 → 2).

Before (master binary, fair local baseline):

before — soldeer-closure edges

After (this branch):

after — immediate dependencies only

QA

  • Discriminating tests: 12 new tests ship with the fix, covering every
    acceptance case: closure entry dropped, direct entry kept,
    direct-AND-transitively-reachable entry kept (NOT-transitive-reduction),
    unknown-tree repo handled (typed, flagged), blockers unchanged vs closure
    edges, AST-vs-comment discrimination, grammar-rejected file fallback,
    version-vs-name-extension prefix matching, and the two survivor-killers
    below.
  • Mutations applied: 18 authored over the new logic (sol_import_paths,
    imported_prefixes, import_package_matches, immediate_deps,
    resolve_node_deps) — one behavior each. Probe pass 1: 16 killed, 2
    survived. Survivors got a discriminating test each and were re-probed to
    KILLED; final matrix 18/18 killed, 0 no-run, 0 harness errors, with the
    suite's own tally (233 baseline / 235 after the new tests) proving every run
    ran:
    • M04 (lexer fallback keeps collecting past the import's ;) — killed by
      the_lexer_fallback_stops_at_the_imports_semicolon: a grammar-rejected
      fixture with import X; followed later by the string
      "phantom-9.9.9/x.sol", which must NOT be collected.
    • M07 (empty prefixes enter the set) — killed by
      an_empty_or_absolute_import_contributes_no_prefix: import ""; and
      import "/abs/X.sol"; contribute nothing.
      Attribution is by construction: the 233-test suite let both live; the only
      delta in the killing suite is the two new tests.
  • Oracle: the org's versioned-import convention (a direct use appears as
    import … from "<package>-<version>/…" in the consumer's own files) and
    live repos, not the implementation: rain.verify / rain.merkle /
    rain.erc4626.words / st0x.atomic-bridge clones grepped independently, plus
    the fair master-binary scan for the graph-level deltas above.
  • Category check: the ask was immediate-only edges decided from data, with
    the named knock-ons checked — blockers (test + live proof above),
    standsOn() tap-highlight (walks the same g.edges transitively; unchanged
    argument, verified against the render), stale-pin Dependency graph ignores pinned versions — a stale pin's transitive closure can differ from what's drawn #79 semantics on surviving
    edges (manifest remains the source; visible in the renders' stale lists),
    and the unreadable-tree state (typed, mirrored from the unparseable-manifest
    state, mutants M12/M16/M17/M18). Transitive reduction explicitly rejected
    and tested against.

Suite: nix develop -c cargo test -p roh-scan → 235 passed / 0 failed.
pre-commit run --all-files clean (rainix static toolchain = CI).
nix run .#dashboard-test → 264 passed / 0 failed.

Summary by CodeRabbit

  • Improvements
    • Dependency graphs now show only immediate, directly imported dependencies.
    • Solidity imports are detected more reliably, including fallback handling for files that cannot be fully parsed.
    • Vendored and generated files are excluded from dependency analysis.
    • Unknown dependency states are clearly preserved when manifests or repository trees cannot be read.
    • Updated scan warnings and graph summaries provide clearer explanations for incomplete dependency information.
  • Bug Fixes
    • Prevented closure-only and stale dependency entries from creating incorrect graph relationships.

claude and others added 3 commits August 19, 2026 04:33
…closure

Soldeer does not resolve transitive dependencies, so org manifests declare
the FULL closure and graph_edges() drew every entry as a direct edge. An
edge now survives only when the consumer's own code (outside vendored
trees) imports the dependency's package prefix, read from the same shallow
clone the scan already takes per Foundry repo — zero extra fetches.

- graph::imported_prefixes / import_package_matches / immediate_deps:
  AST-parsed import extraction (lexer fallback for grammar-rejected files),
  versioned-prefix matching that never splits package names on hyphens,
  and the closure filter with an explicit deps-unknown state.
- A readable manifest with an unreadable tree mirrors the unparseable-
  manifest state: depsKnown=false, no silent keep-all/drop-all, logged.
- Blockers are reachability over the dep set and closure entries are by
  definition reachable through direct ones, so blocker sets are unchanged;
  asserted in tests rather than assumed.
- Stale-pin data (#79) on surviving edges still comes from the manifest.
- site/audit.html: the depsKnown=false label now names both causes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…'s semicolon; empty and absolute import targets name no package

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@thedavidmeister thedavidmeister self-assigned this Aug 19, 2026
@coderabbitai

coderabbitai Bot commented Aug 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b3ef747f-459d-4f23-b821-03fbbc560cc8

📥 Commits

Reviewing files that changed from the base of the PR and between a9fd081 and 44026b7.

📒 Files selected for processing (4)
  • plugins/rain-org-health-check/roh-scan/src/graph.rs
  • plugins/rain-org-health-check/roh-scan/src/main.rs
  • plugins/rain-org-health-check/roh-scan/src/untested.rs
  • site/audit.html

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


Walkthrough

The scanner now extracts Solidity imports, filters them to own non-vendored files, and builds graph edges only for immediate manifest dependencies. It preserves unknown states for unreadable manifests or trees and updates tests and audit messaging.

Changes

Dependency graph resolution

Layer / File(s) Summary
Import extraction and package matching
plugins/rain-org-health-check/roh-scan/src/graph.rs, plugins/rain-org-health-check/roh-scan/src/untested.rs
Solidity imports use AST parsing with lexer fallback. Relative, empty, absolute, and vendored imports are excluded. Package and version prefixes are matched explicitly.
Immediate edge construction and graph validation
plugins/rain-org-health-check/roh-scan/src/graph.rs
Dependency closures are reduced to immediate imported edges. Version comparisons, stale-pin metadata, blocker results, and unreadable-tree states remain covered by tests.
Scan resolution and reporting
plugins/rain-org-health-check/roh-scan/src/main.rs, site/audit.html
The scan distinguishes resolved, unparseable, and unreadable dependency states. It collects imports during the shared clone walk, stores dependency knowledge explicitly, emits warnings, and updates audit messages.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to 44026

The change narrows audit-graph edges to direct package usage, with the supplied verification checks passing. No actionable merge-blocking risk remains beyond normal review.

Sequence Diagram(s)

sequenceDiagram
  participant ShallowCloneWalk
  participant resolve_node_deps
  participant imported_prefixes
  participant RepoResult
  ShallowCloneWalk->>imported_prefixes: collect Solidity import prefixes
  ShallowCloneWalk->>resolve_node_deps: provide manifest state and import prefixes
  resolve_node_deps->>RepoResult: store immediate dependencies and deps_known
  resolve_node_deps->>ShallowCloneWalk: emit warning for unreadable trees
Loading

Possibly related PRs

Suggested reviewers: claude

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely states that the audit graph now uses immediate dependencies instead of the Soldeer dependency closure.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch audit-graph-immediate-deps

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

thedavidmeister added a commit that referenced this pull request Aug 19, 2026
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@thedavidmeister
thedavidmeister merged commit a38ca1d into master Aug 19, 2026
9 checks passed
thedavidmeister pushed a commit that referenced this pull request Aug 20, 2026
Master took #176 ([external.package] table), #177 (immediate deps in the
audit graph) and #178 (package names resolve manifest-first with a fallback
to the release workflow's `soldeer-package:` input; unresolvable renders
UNKNOWN via packageKnown, rainix#335) since the last sync. Resolutions:

- main.rs `fetch_inputs`: both sides kept — master's shape (build RepoInputs
  first, then one registry lookup keyed on `inputs.package()`, the
  manifest-or-workflow resolution) composed with this branch's typed
  `foundry.lock`/`.gitmodules` reads (`RepoFile`), which stale-foundry-lock
  needs so a failed fetch cannot read as an absence.
- untested.rs: this branch's hoisted `pub const VENDOR_DIRS` (shared with
  consumers) kept, master's doc addition about `graph::imported_prefixes`
  kept above it, master's now-duplicate function-local const dropped.
- SKILL.md findings table: master's #178 rewording of `soldeer-unpublished`
  (name from the release-metadata table OR the release workflow) plus this
  branch's `stale-foundry-lock` row appended; denofmt realigned the columns.
- site/health.json: master's side — the hourly scan cron owns that file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants