Skip to content

Re-bless the scenario gateway and diff its numbers - #640

Merged
dfalster merged 2 commits into
developfrom
fix/rebless-scenario-baseline-639
Aug 26, 2026
Merged

dfalster merged 2 commits into
developfrom
fix/rebless-scenario-baseline-639

Conversation

@dfalster

Copy link
Copy Markdown
Member

scenario_baseline.rds matched no build since #583, so
PLANT_RUN_SCENARIOS=1 failed on an unmodified develop and every PR
that ran it inherited a failure it had not caused.

Bisected: phylloptim 0.2.0 -> 0.6.0 (in #633) moves R0 by 8x-32x on five
scenarios and takes S01 across R0 = 1; #619 moves five again on top. Not
#590. Everything else since is inert. Re-blessed on the develop tip as
its own commit, table in NEWS. The gateway now also diffs
offspring_production (SCENARIO_TOL, default 1e-3) and prints the
whole table on failure.

Closes #639.

dfalster and others added 2 commits August 27, 2026 08:06
Recorded on a clean develop tip (6070831), birth-date coordinate,
max_patch_lifetime = 100. Bisection and the per-scenario table are in
NEWS.md and issue #639.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
offspring_production is diffed at a relative tolerance (SCENARIO_TOL,
default 1e-3) alongside observed and persists, and any failure prints
the full per-scenario table. NEWS carries the bisection behind the
re-blessed baseline in the previous commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dfalster

Copy link
Copy Markdown
Member Author

Bisection

One build per commit, each against the phylloptim/odelia sha that commit pins (installed from a git archive of the pinned sha into a scratch library, so no sibling working tree leaks in). macOS arm64, R 4.6.1, -O2 via pkgbuild::compile_dll(debug = FALSE), max_patch_lifetime = 100, SCENARIO_WORKERS = 8. Worker count does not enter the numbers — the scenarios are independent runs.

offspring_production:

scenario baseline #583 133499b8 #606 39c0c657 #633 b374ce3b #619 745dd600 develop 60708313
S01 1.488721 1.488721 1.488721 0.2725883 0.4797763 0.4797763
S02 9.963559e-2 9.963559e-2 9.963559e-2 2.784370e-3 1.153315e-2 1.153315e-2
S03 5.498561e-12 5.498561e-12 5.498561e-12 1.745197e-10 1.665416e-10 1.665416e-10
S04 4.745912e-13 4.745912e-13 4.745912e-13 1.357792e-11 1.317815e-11 1.317815e-11
S05 5.888102e-13 5.888102e-13 5.888102e-13 5.052547e-12 2.470008e-11 2.470008e-11
S06 3.261466e-13 3.261466e-13 3.261466e-13 5.082346e-12 6.570073e-12 6.570073e-12
S07 1415.200 1415.200 1415.200 837.6889 810.7050 810.7050
S08 3.230258e-14 3.230258e-14 3.230258e-14 2.711131e-13 2.025578e-14 2.025578e-14

Step ratios:

scenario -> #633 -> #619 net
S01 0.183 1.76 0.322
S02 0.0280 4.14 0.116
S03 31.74 0.954 30.29
S04 28.61 0.971 27.77
S05 8.581 4.889 41.95
S06 15.58 1.293 20.14
S07 0.592 0.968 0.573
S08 8.393 0.0747 0.627

persists changes once, at #633: S01 TRUE -> FALSE. Nothing else moves on either flag anywhere in the range.

b3472d43 (#608) was measured rather than b374ce3b (#633) — the two commits between them, #607 and #608, touch only NEWS.md, test-leaf.r and notes/, so they compile to the same library. The #633 column is that measurement.

Which half of #633

Rebuilt b374ce3b against phylloptim 0.6.0 with odelia held at 0.2.1 (the LinkingTo floor relaxed for the probe; nothing else touched). All eight scenarios identical to the odelia-0.3.1 build, so odelia 0.2.1 -> 0.3.1 is inert on the gateway and the whole move is phylloptim 0.2.0 -> 0.6.0.

That fits odelia's two behavioural changes in the range: stop_domain step rejection (odelia#56) had no declared domain to act on until #619 added Patch::ode_state_valid, and rejecting a non-finite error estimate (odelia#54) only bites a step that was already diverging.

Why #590 looked like the cause and is not

Provenance of the old baseline

git_commit = 99e44db6, git_dirty = TRUE. It could not have been produced by that commit as committed: there, run_scenarios() still defaulted to control() (height) and scenario_run_metadata() had no node_density_in_birth_date field to write. The tree that produced it is the one that landed as 133499b8, which is why that is the control and reproduces it exactly.

Verification of the new test

Against the stale baseline it now reports, in full:

Scenario outcomes changed vs baseline (re-bless via `make bless-scenarios` if intended): persists S01: TRUE -> FALSE | offspring_production moved on 8/8 scenario(s) (tol 0.001)
id         baseline        current        rel
S01        1.488721      0.4797763     -0.678 *
S02      0.09963559     0.01153315     -0.884 *
S03    5.498561e-12   1.665416e-10       29.3 *
S04    4.745912e-13   1.317815e-11       26.8 *
S05    5.888102e-13   2.470008e-11       40.9 *
S06    3.261466e-13   6.570073e-12       19.1 *
S07          1415.2        810.705     -0.427 *
S08    3.230258e-14   2.025578e-14     -0.373 *

which reproduces the table in #639 exactly. Against the re-blessed baseline it passes.

Tolerance

SCENARIO_TOL defaults to 1e-3. It is not measured against cross-platform drift — the gateway is opt-in and has only ever been run on macOS arm64 — so it is set from the other end: small enough to catch the smallest move anyone has wanted to see (#633's +0.36% on the reference run), large enough to sit well clear of floating-point reordering. If a platform proves noisier, raise it there rather than widening the default; the two classifications are the part that is meant to be platform-stable.

@dfalster
dfalster merged commit c64b8d9 into develop Aug 26, 2026
3 checks passed
@dfalster
dfalster deleted the fix/rebless-scenario-baseline-639 branch August 26, 2026 22:44
dfalster added a commit that referenced this pull request Sep 7, 2026
Every pinned number in #617 was measured against a develop that has since
moved (bounded storage #619, TF24_floor #635, re-blessed gateway #640), so
they are re-measured here rather than carried across.

scientific_version is 10, not 9: develop took 9 for #619.

The reduction test now pins develop's own values, which is what "setting
D_c/theta_c/L_tip to 0 and K_s to 1 recovers the previous model" has to mean
after the merge -- and it passes unmodified.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scenario gateway baseline predates #590 and is red on develop

1 participant