Skip to content

Populate the Go module graph without the toolchain - #8544

Draft
knutwannheden wants to merge 3 commits into
mainfrom
go-dependency-insight-marker
Draft

Populate the Go module graph without the toolchain#8544
knutwannheden wants to merge 3 commits into
mainfrom
go-dependency-insight-marker

Conversation

@knutwannheden

@knutwannheden knutwannheden commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Motivation

GoResolutionResult fell back to go.sum whenever parse-time go list / go mod graph failed. go.sum is a hash inventory of the module graph rather than a build list — it deliberately records versions MVS rejected — so anything reading resolvedDependencies reported modules that are not in the build, with no way to detect that it was in that state. This left Go the weakest of the ecosystems on graph availability: rewrite-maven resolves the graph itself via MavenPomDownloader with no mvn process, rewrite-javascript reads it from a lockfile committed to the repo, and only Go depended on an external process succeeding at parse time.

  • The prerequisite for a Go DependencyInsight (moderneinc/customer-requests#3017) is that transitive dependency questions have a trustworthy answer, or a truthful "I don't know".

Summary

  • resolvedDependencies now means the resolved build list, derived in priority order from the toolchain, vendor/modules.txt, or the main module's require block. The last leans on Go 1.17+ module graph pruning, which records an indirect require for every module providing a transitively imported package and so makes the expanded require block a build list — derivable with no toolchain, no module cache and no network. go.sum joins in by (path, version) to supply hashes; rows matching no build-list member are versions MVS rejected and stay unselected.
  • Graph edges are enriched onto whatever build list was derived by reading each member's own go.mod from the module cache at cache/download/<escaped>/@v/<version>.mod, whose requires are that module's edges — the same set go mod graph prints. Because the build list has already selected every version, this needs no version resolution.
  • Two fields make the state legible instead of inferred: resolutionSource (TOOLCHAIN / VENDOR / GO_MOD / GO_SUM_ONLY) and per-dependency selected, with hasBuildList() and hasGraph() helpers. Recipes use the helpers rather than comparing constants, so further sources can be added without touching call sites. hasGraph() reads the data rather than resolutionSource, because edges are an enrichment any source can carry — a vendored module with a warm cache has both.
  • GoModParser.withSumHashes merges go.sum into the build list rather than replacing it wholesale, which was safe only while the Go side always returned it empty. Its sibling lookup resolves through relativeTo, since a repo-relative source path is an identifier rather than a filesystem location.

Only a pre-pruning module now falls back to a bare hash inventory, and it reports GO_SUM_ONLY rather than presenting the inventory as a build list.

Test plan

  • 21 new Go parser tests: pruning gate and pre-1.17 fallback, replace binding (versioned, wildcard, non-matching), go.sum hash join, unselected rejected versions, vendor grammar including ## explicit / => / per-module go version, module-cache path and version escaping, cache misses leaving deps nil rather than empty, and cache enrichment over a vendored build list
  • Codec round-trip mutation-checked — deleting the two send lines fails it with a wire desync rather than passing on zero values
  • Full Go suite green (go test ./...)
  • 175 rewrite-go Java tests across unit and integTest, 0 failures, 0 errors — including MarkerRoundTripTest over the real Java → Go → Java RPC path with assertions on both new fields. One pre-existing environmental skip (Assumptions.assumeTrue(recipesPath != null))

…lable

`GoResolutionResult.resolvedDependencies` previously held whatever go.sum
recorded whenever parse-time toolchain resolution failed. go.sum is a hash
inventory of the module graph, not a build list: it deliberately includes
versions MVS rejected, so a recipe reading it reports modules that are not in
the build at all.

Under Go 1.17+ module graph pruning the main module's expanded `require` block
names every module providing a transitively imported package, so it is the
build list — derivable with no toolchain, no module cache and no network.
`resolvedDependencies` now means the build list, with go.sum joined in by
(path, version) for hashes and rows matching nothing kept unselected.

Two fields let recipes tell these states apart instead of inferring them:

- `ResolutionSource` (TOOLCHAIN / GO_MOD / GO_SUM_ONLY) with `hasBuildList()`
  and `hasGraph()`. Recipes use the helpers, so later sources can be added
  without touching call sites.
- `ResolvedDependency.selected`, distinguishing build-list members from go.sum
  rows for versions that lost.

`GoModParser.withSumHashes` replaced `resolvedDependencies` wholesale, which
was harmless only while the Go side always returned it empty. It now merges.
Its sibling lookup also resolved a repo-relative source path against the
process working directory, so a test run would read whatever go.sum sat there —
for rewrite-go's own suite, its own, injecting `github.com/creack/pty` into
fixtures that never mention it. `GoModConformanceTest` already routed around
this with a virtual source path. The lookup now resolves through `relativeTo`,
which is what turns a repo-relative identifier back into a location, and skips
the read when no absolute path can be formed.
A build list derived from go.mod carries no edges, so transitive dependency
questions still needed the toolchain. Two offline sources close that gap.

vendor/modules.txt is authoritative for a vendored build and is the only
offline source of the package-to-module map — in Go an import path is not a
module coordinate, so that mapping cannot be recovered from go.mod. It ranks
above the go.mod-derived build list, below the toolchain.

The module cache already holds each dependency's own go.mod at
cache/download/<escaped>/@v/<version>.mod, and its requires are that module's
edges — the same set `go mod graph` prints. Because the build list has already
selected every version, attaching edges needs no version resolution. Modules
absent from a partially warm cache keep nil edges, which is distinct from an
empty slice: no edges known, rather than no dependencies.

Edges are an enrichment over whatever build list was derived, not a source of
one: a vendored module with a warm cache has both. `hasGraph()` therefore reads
the data rather than `resolutionSource`, and the cache is not an enum constant.
`resolutionSource` means only where the build list came from.
- The module cache escapes versions as well as paths, so a module at a version
  containing uppercase (v1.0.0-RC1 is stored as v1.0.0-!r!c1.mod) never
  resolved and silently read as having no known edges.
- `DeriveBuildList` ignored `replace`, so a forked dependency landed selected
  under its original coordinate with no ReplacePath, inverted from what the
  toolchain path reports. `exclude` stays unapplied: under pruning the require
  block already reflects it.
- `hasGraph()` iterated `resolvedDependencies` unguarded while every other
  access in the class treats it as nullable, including the receive path, where
  `receiveList` yields null for a DELETE.
- A marker predating `resolutionSource` arrives with it unset; `Enum.valueOf`
  on the empty string failed the whole RPC exchange rather than degrading to
  GO_SUM_ONLY.
- GOPATH is a list, and the module cache lives under its first entry.
- `parseSumSibling` skipped the read for any non-absolute path, dropping hashes
  when a caller passed a relative project root. A supplied root states where the
  repo is, so it resolves against the working directory; a bare source path
  without one remains an identifier and is not a location.
- GO_MOD and VENDOR no longer claim to have no graph edges, since cache
  enrichment applies to both.
@knutwannheden
knutwannheden force-pushed the go-dependency-insight-marker branch from b09850d to f3853cd Compare August 19, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant