Skip to content

test(bb): run every root rollup check nightly, including the pinned vk - #25165

Open
AztecBot wants to merge 1 commit into
nextfrom
cb/root-rollup-tests-nightly
Open

test(bb): run every root rollup check nightly, including the pinned vk#25165
AztecBot wants to merge 1 commit into
nextfrom
cb/root-rollup-tests-nightly

Conversation

@AztecBot

Copy link
Copy Markdown
Collaborator

Summary

#24799 moved the heavy recursion suites to nightly but kept PinnedVKRootRollup per merge as the cheap change-detector for the root rollup circuit. The first merge-queue run after it landed shows it is not cheap.

From run 31409209377 (ci-full-no-test-cache, head c38ea15f, test-engine log 6b948df91b0fd544):

dsl_tests HonkRecursionConstraintTestWithoutPredicate/2.PinnedVKRootRollup   261s

That is the longest barretenberg test in the merge queue, against a default 600s per-test timeout — for a circuit whose full checks had already been deferred to nightly.

Comparing against the last pre-#24799 merge-queue run (31395115064, head 02217d37, test-engine log b96eb5be8d914778), the per-merge cost of the root rollup circuit went:

before after
…/2.GateCountRootRollup…/2.PinnedVKRootRollup 29s 261s
…/2.GenerateVKFromConstraints 184s nightly
…/2.Tampering 57s nightly
total per merge 270s 261s

A wash — and with a longer critical path, since the work is now concentrated in one serial test rather than spread across three. The cost is constructing the ~6.35M-gate circuit at all, not the checks layered on top of it, so there is no cheaper per-merge detector to keep behind.

What this does

Drops the *.PinnedVKRootRollup carve-out in runs_nightly_only, so the whole of HonkRecursionConstraintTestWithoutPredicate/2 — pinned vk hash, pinned gate count, GenerateVKFromConstraints, Tampering — is emitted by test_cmds_nightly at CPUS=16:MEM=32g:TIMEOUT=60m.

Unchanged, so the smaller recursion circuits keep their per-merge detectors:

  • the *.GateCount* carve-out — HonkRecursionConstraintTestWithPredicate/*.GateCountSingleHonkRecursion and ChonkRecursionConstraintTest.GateCountChonkRecursion still run per merge
  • the debug-build carve-out for WithoutPredicate/1.GenerateVKFromConstraints, the only case exercising the debug-only native_verification_debug path

Also removes the HonkRecursionConstraintTestWithoutPredicate/2.* resource-prefix branch. Nightly tests get a fixed prefix before that code is reached, and no /2 case is emitted per merge any more, so the branch became unreachable; leaving it would have been a comment asserting a per-merge policy that no longer exists. The three comments that still described the pinned vk as the per-merge cover are updated.

The trade-off

The pinned gate count and vk hash become nightly-only, so a PR that changes the root rollup circuit merges green and the nightly goes red the next morning — the failure lands on next rather than on the PR that caused it. That is the deliberate cost of not paying 261s per merge for a circuit whose real checks already run nightly.

Worth knowing: the nightly job itself has never run. barretenberg-nightly-slow-tests.yml landed with #24799 at 14:33 UTC today and its cron is 0 6 * * *, so total_count for that workflow is still 0. It has workflow_dispatch — a manual trigger before this merges would validate the nightly emission end to end.

Testing

  • Sourced nightly_only_tests + runs_nightly_only into a harness and classified the real test names taken from the two merge-queue logs above, under both clang20 and a debug preset:
    • all three WithoutPredicate/2 cases → nightly, in both presets
    • GateCountSingleHonkRecursion, GateCountChonkRecursion → per merge
    • WithoutPredicate/1.GenerateVKFromConstraints → per merge in debug only
    • AvmRecursiveTests.TranscriptOperations, IPARecursiveTests.*, ChonkTests.Basic → per merge, unaffected
  • bash -n barretenberg/cpp/bootstrap.sh
  • C++ changes are comment-only.

Created by claudebox · group: slackbot · requested by Sergei · Slack thread

Moving the heavy recursion suites to nightly kept PinnedVKRootRollup per merge as the
cheap change-detector for the root rollup circuit. The first post-merge merge-queue run
shows it is not cheap: it took 261s, making it the longest barretenberg test in the merge
queue, against a default 600s per-test timeout.

The cost is constructing the ~6.35M-gate circuit at all, not the checks layered on top, so
there is no cheaper per-merge detector to keep behind. Per merge that circuit went from 29s
(GateCountRootRollup) to 261s while 241s of vk-independence and tampering work moved to
nightly — a wash for the merge queue, and a longer critical path.

Drop the PinnedVKRootRollup carve-out so the whole of
HonkRecursionConstraintTestWithoutPredicate/2 is emitted by test_cmds_nightly:
the pinned vk hash and gate count now run there alongside GenerateVKFromConstraints and
Tampering. The trade is that a change to the root rollup circuit is caught by the next
nightly rather than in the merge queue.

The GateCount carve-out and the debug-build carve-out for
WithoutPredicate/1.GenerateVKFromConstraints are unchanged, so the smaller recursion
circuits keep their per-merge detectors. The WithoutPredicate/2 resource-prefix branch is
removed with it: nightly tests get a fixed CPUS=16:MEM=32g:TIMEOUT=60m, so the branch could
no longer be reached.
@AztecBot AztecBot added ci-draft Run CI on draft PRs. ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure claudebox Owned by claudebox. it can push to this PR. labels Aug 10, 2026
@iakovenkos
iakovenkos self-requested a review August 10, 2026 17:51
@iakovenkos iakovenkos removed ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure ci-draft Run CI on draft PRs. labels Aug 10, 2026
@iakovenkos
iakovenkos marked this pull request as ready for review August 10, 2026 17:51
@iakovenkos
iakovenkos enabled auto-merge August 10, 2026 17:51
@iakovenkos
iakovenkos added this pull request to the merge queue Aug 10, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 10, 2026
@iakovenkos
iakovenkos added this pull request to the merge queue Aug 11, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claudebox Owned by claudebox. it can push to this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants