Populate the Go module graph without the toolchain - #8544
Draft
knutwannheden wants to merge 3 commits into
Draft
Conversation
…lable `GoResolutionResult.resolvedDependencies` previously held whatever go.sum recorded whenever parse-time toolchain resolution failed. go.sum is a hash inventory of the module graph, not a build list: it deliberately includes versions MVS rejected, so a recipe reading it reports modules that are not in the build at all. Under Go 1.17+ module graph pruning the main module's expanded `require` block names every module providing a transitively imported package, so it is the build list — derivable with no toolchain, no module cache and no network. `resolvedDependencies` now means the build list, with go.sum joined in by (path, version) for hashes and rows matching nothing kept unselected. Two fields let recipes tell these states apart instead of inferring them: - `ResolutionSource` (TOOLCHAIN / GO_MOD / GO_SUM_ONLY) with `hasBuildList()` and `hasGraph()`. Recipes use the helpers, so later sources can be added without touching call sites. - `ResolvedDependency.selected`, distinguishing build-list members from go.sum rows for versions that lost. `GoModParser.withSumHashes` replaced `resolvedDependencies` wholesale, which was harmless only while the Go side always returned it empty. It now merges. Its sibling lookup also resolved a repo-relative source path against the process working directory, so a test run would read whatever go.sum sat there — for rewrite-go's own suite, its own, injecting `github.com/creack/pty` into fixtures that never mention it. `GoModConformanceTest` already routed around this with a virtual source path. The lookup now resolves through `relativeTo`, which is what turns a repo-relative identifier back into a location, and skips the read when no absolute path can be formed.
A build list derived from go.mod carries no edges, so transitive dependency questions still needed the toolchain. Two offline sources close that gap. vendor/modules.txt is authoritative for a vendored build and is the only offline source of the package-to-module map — in Go an import path is not a module coordinate, so that mapping cannot be recovered from go.mod. It ranks above the go.mod-derived build list, below the toolchain. The module cache already holds each dependency's own go.mod at cache/download/<escaped>/@v/<version>.mod, and its requires are that module's edges — the same set `go mod graph` prints. Because the build list has already selected every version, attaching edges needs no version resolution. Modules absent from a partially warm cache keep nil edges, which is distinct from an empty slice: no edges known, rather than no dependencies. Edges are an enrichment over whatever build list was derived, not a source of one: a vendored module with a warm cache has both. `hasGraph()` therefore reads the data rather than `resolutionSource`, and the cache is not an enum constant. `resolutionSource` means only where the build list came from.
- The module cache escapes versions as well as paths, so a module at a version containing uppercase (v1.0.0-RC1 is stored as v1.0.0-!r!c1.mod) never resolved and silently read as having no known edges. - `DeriveBuildList` ignored `replace`, so a forked dependency landed selected under its original coordinate with no ReplacePath, inverted from what the toolchain path reports. `exclude` stays unapplied: under pruning the require block already reflects it. - `hasGraph()` iterated `resolvedDependencies` unguarded while every other access in the class treats it as nullable, including the receive path, where `receiveList` yields null for a DELETE. - A marker predating `resolutionSource` arrives with it unset; `Enum.valueOf` on the empty string failed the whole RPC exchange rather than degrading to GO_SUM_ONLY. - GOPATH is a list, and the module cache lives under its first entry. - `parseSumSibling` skipped the read for any non-absolute path, dropping hashes when a caller passed a relative project root. A supplied root states where the repo is, so it resolves against the working directory; a bare source path without one remains an identifier and is not a location. - GO_MOD and VENDOR no longer claim to have no graph edges, since cache enrichment applies to both.
knutwannheden
force-pushed
the
go-dependency-insight-marker
branch
from
August 19, 2026 08:13
b09850d to
f3853cd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
GoResolutionResultfell back to go.sum whenever parse-timego list/go mod graphfailed. go.sum is a hash inventory of the module graph rather than a build list — it deliberately records versions MVS rejected — so anything readingresolvedDependenciesreported modules that are not in the build, with no way to detect that it was in that state. This left Go the weakest of the ecosystems on graph availability:rewrite-mavenresolves the graph itself viaMavenPomDownloaderwith nomvnprocess,rewrite-javascriptreads it from a lockfile committed to the repo, and only Go depended on an external process succeeding at parse time.DependencyInsight(moderneinc/customer-requests#3017) is that transitive dependency questions have a trustworthy answer, or a truthful "I don't know".Summary
resolvedDependenciesnow means the resolved build list, derived in priority order from the toolchain,vendor/modules.txt, or the main module'srequireblock. The last leans on Go 1.17+ module graph pruning, which records an indirect require for every module providing a transitively imported package and so makes the expanded require block a build list — derivable with no toolchain, no module cache and no network. go.sum joins in by(path, version)to supply hashes; rows matching no build-list member are versions MVS rejected and stay unselected.cache/download/<escaped>/@v/<version>.mod, whose requires are that module's edges — the same setgo mod graphprints. Because the build list has already selected every version, this needs no version resolution.resolutionSource(TOOLCHAIN/VENDOR/GO_MOD/GO_SUM_ONLY) and per-dependencyselected, withhasBuildList()andhasGraph()helpers. Recipes use the helpers rather than comparing constants, so further sources can be added without touching call sites.hasGraph()reads the data rather thanresolutionSource, because edges are an enrichment any source can carry — a vendored module with a warm cache has both.GoModParser.withSumHashesmerges go.sum into the build list rather than replacing it wholesale, which was safe only while the Go side always returned it empty. Its sibling lookup resolves throughrelativeTo, since a repo-relative source path is an identifier rather than a filesystem location.Only a pre-pruning module now falls back to a bare hash inventory, and it reports
GO_SUM_ONLYrather than presenting the inventory as a build list.Test plan
replacebinding (versioned, wildcard, non-matching), go.sum hash join, unselected rejected versions, vendor grammar including## explicit/=>/ per-module go version, module-cache path and version escaping, cache misses leavingdepsnil rather than empty, and cache enrichment over a vendored build listgo test ./...)rewrite-goJava tests across unit and integTest, 0 failures, 0 errors — includingMarkerRoundTripTestover the real Java → Go → Java RPC path with assertions on both new fields. One pre-existing environmental skip (Assumptions.assumeTrue(recipesPath != null))