Skip to content

fix: pin the published artifacts, and decode addresses instead of shape-checking them - #77

Merged
moul merged 1 commit into
mainfrom
fix/pin-artifact-and-validate-addresses
Sep 11, 2026
Merged

moul merged 1 commit into
mainfrom
fix/pin-artifact-and-validate-addresses

Conversation

@moul

@moul moul commented Sep 11, 2026 •

Copy link
Copy Markdown
Member

Two gaps in the verification layer, found while checking counterparty addresses against the sheet. Both are in the class this repository exists to prevent: something that every check calls correct, and isn't.

No artifact changes. genbalance.txt.gz, balances.txt.gz, README.md and unrestricted.txt are byte-identical after a full rebuild — total supply still 1,332,999,998.328067 GNOT across 3,262,473 rows. Rebased onto main after #73 and #76 landed.


1. The published .gz is not byte-pinned, and macOS silently writes a different one

mkgenesis/balances.txt.gz is fetched by raw URL from outside this repo — ABOUT.md calls it a public contract. Its bytes depend on which gzip built it:

committed   aaf850fdb1bc9e20ecc9d612  57,243,554 bytes
GNU 1.12    aaf850fdb1bc9e20ecc9d612  57,243,554   ✓
GNU 1.14    aaf850fdb1bc9e20ecc9d612  57,243,554   ✓
Apple 479   7d5486228b7e0f93a40ac57b  57,000,132   ✗  different file, 243 KB smaller

(measured against the pre-#73 artifact; the committed hash has moved since, the behaviour has not)

Same gzip -n, same plaintext, different deflate stream. Nothing here could see it: the golden test decompresses before comparing (golden_test.go compares balances.txt, never the container), crosscheck decompresses too, and CI runs ubuntu-latest, where gzip is always GNU.

The docs made it worse by asserting the opposite in four places — including a gotchas row that gave the wrong diagnosis (you are on a build without gzip -n, when the Makefile always passes -n), and docs/reproduce.md Level 1 telling an external verifier git status # should be clean followed by "If yours differ, that is worth an issue." Someone on macOS doing exactly the independent verification this repo is for would conclude the genesis had been tampered with.

This PR:

  • SHA256SUMS, committed and asserted by a new make checksums, wired into make verify and CI. make supply still prints the hashes — but printing is not verifying, and nothing failed when they changed. This is also the only check that ever sees the gzip container.
  • make tools refuses to run on a non-GNU gzip, naming the problem and the fix instead of letting it surface as a confusing diff.
  • make checksums-update for when the artifacts are meant to change.
  • Corrected the four claims in Makefile, docs/reproduce.md (×3) and mkgenesis/ABOUT.md.
$ make tools
WRONG GZIP: Apple gzip 479
  balances.txt.gz only reproduces with GNU gzip; this one writes a different container.
  macOS: brew install gzip, then put it first on PATH:
    export PATH="$(brew --prefix)/opt/gzip/bin:$PATH"
  nix: nix shell nixpkgs#gzip

2. Sheet addresses were shape-checked, never decoded

TestHardcodedAddressesAreValid does real validation — but only of the constants, which change rarely and under review. The hand-written sheets, where counterparty addresses arrive by email and get pasted in days before genesis, got:

  • parseRow() — no address validation at all; everything before = was taken as the address
  • addrRE in unrestricted.go — ^g1[0-9a-z]{38}$, shape only
  • the row regexes in TestGenesisFileTotal / TestInvestorDistributionsMatchFile — shape only

That is backwards from where the risk is. A transposed character preserves the shape and breaks only the checksum. Before this PR, on a branch with a two-character transposition in a real counterparty address:

$ go test ./...
ok  github.com/gnolang/independence-day/allocate
ok  github.com/gnolang/independence-day/mkgenesis

$ gzip -dc mkgenesis/balances.txt.gz | grep pvekenc
g1pvekenc72l8d05nu5n0ym0v5za05fz80yepdpq=100000000ugnot
$ grep pvekenc mkgenesis/unrestricted.txt
g1pvekenc72l8d05nu5n0ym0v5za05fz80yepdpq

Full suite green, invalid address in both the shipped artifact and the §126 whitelist. After:

$ cd mkgenesis && go run . build
mkgenesis: non-airdrop.txt:201: address "g1pvekenc72l8d05nu5n0ym0v5za05fz80yepdpq" is not valid bech32:
  invalid checksum (expected (bech32=uqz3l3, bech32m=uqz3l3fuja6n), got yepdpq)

Being precise about severity: this was never silent fund loss. gnoland rejects it downstream — Balance.Parse → crypto.AddressFromBech32 → checksum. But "downstream" meant at the genesis ceremony rather than in CI seconds after the paste. This moves the failure to where it's cheap.

This PR adds mkgenesis/addr.go with addrKey/checkAddr — deliberately the same definition as allocate's addrKey, since both are package main and the helper can't be shared, but "a valid address" must not fork between the stage that produces rows and the stage that merges them. checkAddr also requires canonical g1… spelling, so a cosmos1… form of the right 20 bytes is rejected too: consumers compare these as strings.

Wired into the hand-written sheet readers, the whitelist generator, and TestGenesisFileTotal.

The one deliberate asymmetry

accumulate validates the hand-written sheets and not genbalance. Measured, not assumed: decoding all 3.26M rows costs ~3.4s, which would land on every build and every golden test. Those rows are produced by allocate's addrKey and are canonical by construction.

The shipped artifact is still decoded in full — once, in TestGenesisFileTotal, which already walks every row. That moves the allocate suite from ~10s to ~18s, which seemed the right place to spend it. TestGenbalanceIsNotRevalidated documents the asymmetry so it doesn't read as an oversight.

Test fixtures

The old placeholders (g1aaa, g1shared, …) are not valid bech32, so they no longer parse — which is the check working. Replaced with real 20-byte payloads (0x11…, 0x22…, 0x33… repeated) that have correct checksums, plus testAddrTransposed for the regression tests.

Verification

make verify
  allocate  ok (17.8s)   mkgenesis ok (5.6s)
  crosscheck                        Balance files match.
  checksums  allocate/genbalance.txt.gz: OK
             mkgenesis/balances.txt.gz:  OK
  supply     3262473 rows, 1332999998328067 ugnot (1332999998.328067 GNOT)

CI green, including the new checksum step. Worth noting how the first CI run failed: I had branched before #73 merged, so SHA256SUMS described the older artifacts and make checksums caught the staleness immediately — which is the check doing exactly its job, on its first outing.

SHA256SUMS does have to be regenerated (make checksums-update) in any PR that legitimately changes an artifact, and it will conflict on rebase when two such PRs are in flight. That is the intended cost: a conflict is a prompt to re-verify, where today the file just changes.

Full rebuild from scratch (allocate → mkgenesis re → readme) leaves every committed artifact untouched; git status shows only the source changes in this PR. go vet ./... clean.

Not in this PR

gnolang/gno's misc/deployments/gnoland1/gen-genesis.sh:45 still pins BALANCES_GZ_URL to 9dec38a4 (2026-03-12, 27 commits back = 1.002B GNOT vs today's 1.333B) and does no sha256 check on the download. Correct for reproducing gnoland1's historical genesis; a hazard if reused for mainnet. SHA256SUMS here gives that script something to verify against — the other half belongs in a gnolang/gno PR.

@moul
moul force-pushed the fix/pin-artifact-and-validate-addresses branch from 311ab66 to 5190b1e Compare September 11, 2026 09:45
@moul
moul merged commit b8e4798 into main Sep 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant