This repository was archived by the owner on Sep 8, 2026. It is now read-only.
contributor-rewards: add golden tests covering reward values - #414
Merged
Merged
Conversation
Add a trimmed mainnet-beta snapshot fixture and the script that produces it, for a future Shapley golden test. The only previously committed snapshot (testnet) produces zero reward for every operator, so no golden test can assert a real value against it. Exact Shapley computation is O(2^n) in the operator count, so the fixture keeps only the contributors and cities that dominate device count on the full mainnet-beta capture, which keeps the sub-network connected and the computation fast. It is not a byte-size trim of the full topology. Also migrate the access pass status value Expired to ExpiredDeprecated when loading a snapshot captured before doublezero-serviceability PR #3831 renamed that enum variant, without which the full mainnet-beta capture fails to deserialize.
Clamp near-zero Shapley entries in assert_close so cancellation noise around 1e-12 does not sit below its own comparison gate. Fix UPDATE_GOLDEN to regenerate only on the value "1", not any set value. Rename the mn-beta fixture and goldens to mainnet-beta, fix the wrong regeneration path in make-fixture.py, and apply the remaining minor cleanups from the review.
martinsander00
approved these changes
Aug 27, 2026
bgm-malbeclabs
enabled auto-merge (squash)
August 27, 2026 21:24
bgm-malbeclabs
added a commit
to malbeclabs/doublezero
that referenced
this pull request
Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
contributor-rewardshad 113 unit tests and 8 integration tests, and not one asserted a reward figure. This crate decides what contributors are paid.tests/goldens/mainnet-beta-epoch-129-trimmed.json, plusmake-fixture.pythat regenerates it from a full capture.src/ingestor/types.rsgains anAccessPassStatus::ExpiredtoExpiredDeprecatedcompat migration, alongside the migrations already inapply_serviceability_json_compat_migrations. Without it the epoch-129 capture cannot be deserialized at all. Verified against upstreamaccesspass.rs:112, same discriminant retained. The fixture holds 10Expiredpasses, so it is genuinely exercised.Tracks malbeclabs/infra#2392. Prompted by the monorepo migration design (malbeclabs/doublezero#4239), whose step 4 merges this crate into a shared Cargo workspace. That merge can change resolved dependency versions, and nothing today would notice if reward figures moved as a result.
Why the fixture is a trimmed subnetwork
Two constraints forced this shape, and both were measured rather than assumed.
The existing
tests/testnet_snapshot.jsonproduces 0.0 for every operator in every city. That is genuine output, not a harness bug: the Shapley machinery works (evaluator.rs's own test asserts nonzero proportions), the input assembly matchesPreparedData::from_snapshotbuilder for builder, andtest_pvt_links.rspins real latencies from the same snapshot. Those private links are simply no faster than public internet on those routes, so no operator adds value. Pinning zeros would give a test that passes forever and detects nothing.The full mainnet capture goes the other way. Exact Shapley is O(2^n) in operators, and 14 operators across 30 cities was killed after 49 CPU-minutes without finishing.
So the fixture keeps 4 operators and 6 cities, which produces nonzero rewards in 0.3 seconds. The coverage limit is real and worth stating: a regression reachable only through the dropped topology goes unseen. That is a much smaller gap than the zero coverage it replaces.
Why values use a tolerance rather than exact equality
Structure is asserted exactly: operator set, operator ordering, city set, per-city ordering, all counts. Values and proportions use a
1e-12relative tolerance.The computation is deterministic on one machine, verified under
RAYON_NUM_THREADSof 1, 2 and 16 and bit-identical in a release build with different codegen and LTO. Bit-identical floating point across architectures is not guaranteed, though, and these goldens were generated on arm64 while CI is x86_64. An exact gate would go permanently red the day CI changes architecture, and a permanently red test is one everybody learns to ignore.assert_closealso skips entries where both sides are below1e-6. Seven of the 24 per-city entries are cancellation noise near1e-12, against a largest real value of2331.52. Gating those at an absolute1e-12would fail on differences that carry no reward information, and the natural response to that red isUPDATE_GOLDEN=1, which also quietly destroys the detector. The clamp requires both sides to be negligible, so a real value collapsing to zero is still caught.Testing Verification
git status. A golden test that cannot fail is the failure mode that matters here, and passing twice does not prove it can.contiguity_bonusby2e-10moves the output by2.5e-10, so a real change in reward math lands 8 or more orders of magnitude above the1e-12gate.RAYON_NUM_THREADS1, 2 and 16, and between debug and release builds.doublezero-serviceabilitysource rather than inferred from the error.just fmt-checkandjust clippyclean.Known follow-ups, deliberately not in this PR
make-fixture.pytruncates sample arrays but does not filter telemetry records to the surviving devices, so 254 of 328 device records are inert. Proven safe to leave: a filtered copy (4.66 MB to 3.33 MB) reproduces the golden exactly, so the filter can be added later with no regeneration. Noted in the script's docstring.[profile.test]opt-level for the workspace. Not needed for these tests at 0.3s, but the two Rust CI jobs currently take 17 and 20 minutes.