Skip to content

test: tripwire for a contextScope that changes across REST versions - #131

Merged
juemerson-at-purestorage merged 1 commit into
dmann000:mainfrom
juemerson-at-purestorage:test/issue-112-contextscope-version-tripwire
Aug 20, 2026
Merged

juemerson-at-purestorage merged 1 commit into
dmann000:mainfrom
juemerson-at-purestorage:test/issue-112-contextscope-version-tripwire

Conversation

@juemerson-at-purestorage

Copy link
Copy Markdown
Collaborator

What this adds

contextScope is one value per endpoint in Data/PfbCapabilityMap.json, computed last-seen-wins
across the whole 2.0-2.28 spec set, with no version dimension. #112 records that as a deliberate
decision resting on a premise: a resource does not migrate between the fleet database and an
individual array, so scope is a property of the resource rather than of the API version.

Nothing tested the premise. If it ever stopped holding, the generator would silently record the
newest reading, the runtime gates would apply it to an array running an older REST version, and no
existing test could see it — the drift suite compares the shipped artifact against generator output,
and both sides would agree.

This adds three cross-version assertions over every cached spec, plus a fourth that needs only a
single version:

Assertion Fires when Why the map cannot absorb it
ValueChange an endpoint declares a different scope in a later version scope is version-dependent after all, so one value per endpoint is wrong
KindGained earlier tokens survive and a new one joins them (FLEET → FLEET|REALM) one scalar scope cannot hold two kinds, and the generator's FLEET-wins rule would quietly resolve it
Withdrawn a later spec still has the operation but drops its override last-seen-wins reverts the endpoint to the array default, changing what the runtime gates enforce
vocabulary any spec declares a domain token outside ARRAY/FLEET see below

The vocabulary assertion is the one most likely to fire first, and the reason it is separate rather
than folded into the others: it needs a single version to declare an unfamiliar token, not two
versions to disagree. tools/Build-PfbCapabilityMap.ps1 sends an unrecognised token to
scope: unknown, and unknown suppresses the kind-vs-scope gate in
Private/Assert-PfbContextSupported.ps1 — so a new domain would not fail loudly, it would stop
validating those endpoints. Set-PfbContext -Kind already accepts TopologyGroup with no spec
vocabulary behind it, so the client half of that gap exists today.

What it does not claim

The real-spec assertions are vacuously green today. All five endpoints that declare an override
— the /presets/workload verbs — declare it only in fb2.28, so there are zero cross-version pairs to
compare. Only the vocabulary assertion has live data behind it.

That is the point of a tripwire, but it creates a problem: a tripwire nobody has seen trip is
indistinguishable from one that cannot. So the comparison is a separate function driven by synthetic
declarations in its own ungated Describe — seven cases proving each finding fires, and proving two
near-misses do not: an operation leaving the API entirely is not a withdrawal, and a version outside
the scanned range is ignored rather than mis-ranked as earliest.

An empty walk is inconclusive, never negative. The 2.17 $ref restructuring makes a walk that
silently returns nothing easy to write, and an empty result reads as "nothing is affected". A
positive control therefore asserts the scan saw the API surface — more than 500 endpoints, at least
one declaration — before any "no findings" assertion is believed. Bounds rather than exact counts, so
a new spec version does not red the file.

Spec files are iterated by filename, never by numeric version literal. The literal 2.20 is the
number 2.2, which opens fb2.2.json — a real file — so a cross-version claim can silently cover a
version it never read.

This does not add the version dimension #112 describes as missing. The omission stands exactly as
recorded; it is now self-monitoring rather than resting on an argument nobody re-examines.

Coverage baseline

winps51 MaxSkipped 268 → 273 for the 5 PS7-gated It blocks, keeping the same +16 headroom the
file's convention uses. Both new Describes added to RequiredDescribes — load-bearing here,
because a vacuously-green block is exactly what a skip ceiling cannot distinguish from one that
stopped running. The synthetic block is listed under both editions (it reads no spec and uses no
PS7-only syntax); the real-spec block under pwsh7 alone.

Verification

Both editions, across this file plus its two neighbours (Build-PfbCapabilityMap.Tests.ps1,
PfbSpecTools.ContextScope.Tests.ps1):

Edition   Pester Status Passed Failed Skipped Container
pwsh 7    6.0.1  Passed     84      0       0 ok
WinPS 5.1 6.0.1  Passed     17      0      67 ok

Test-only: no runtime change, no generator change, no derived-report regeneration, no version bump.

Refs #112 — merging this makes #112 closable as the recorded decision it is, with the tripwire as the
reopen trigger its own text asks for. Left for you to close rather than auto-closing, since the
disposition is separable from whether this test is wanted.

🤖 Generated with Claude Code

…mann000#112)

contextScope is one value per endpoint, computed last-seen-wins across the
whole spec set, with no version dimension. Issue dmann000#112 records that as a
deliberate decision resting on a premise: a resource does not migrate between
the fleet database and an individual array, so scope is a property of the
resource rather than of the API version.

Nothing tested the premise. If it stopped holding, the map would silently
record the newest reading, the runtime gates would apply it to an array running
an older REST version, and no existing test could see it -- the drift suite
compares the shipped artifact against generator output, and both sides would
agree.

Adds three cross-version assertions over every cached spec, plus a fourth that
needs only a single version:

  ValueChange  an endpoint declares a different scope in a later version
  KindGained   an endpoint gains an ADDITIONAL kind (FLEET -> FLEET|REALM),
               which one scalar scope cannot represent and which the
               generator's FLEET-wins rule would quietly resolve
  Withdrawn    a later spec still has the operation but drops its override,
               so last-seen-wins reverts it to the array default
  vocabulary   the declared domain set is still exactly ARRAY and FLEET

The vocabulary assertion is the one most likely to fire first, and the reason
it is separate: Build-PfbCapabilityMap.ps1 sends an unrecognised token to
scope 'unknown', and 'unknown' suppresses the kind-vs-scope gate in
Private/Assert-PfbContextSupported.ps1 -- so a new domain would not fail
loudly, it would stop validating. Set-PfbContext -Kind already accepts
TopologyGroup with no spec vocabulary behind it.

Two things this deliberately does NOT assume. The real-spec assertions are
vacuously green today: all five endpoints that declare an override declare it
only in fb2.28, so there are no cross-version pairs to compare. A tripwire
nobody has seen trip is indistinguishable from one that cannot, so the
comparison is a separate function driven by synthetic declarations in its own
ungated Describe, which runs on both editions and proves each finding fires.
And an empty walk is inconclusive rather than negative -- the 2.17 $ref
restructuring makes a walk that silently returns nothing easy to write -- so a
positive control asserts the scan saw the API surface (>500 endpoints, at
least one declaration) before any "no findings" assertion is believed.

Spec files are iterated by filename, never by numeric version literal: the
literal 2.20 is the number 2.2, which opens fb2.2.json, a real file.

coverage-baseline.psd1: winps51 MaxSkipped 268 -> 273 for the 5 PS7-gated It
blocks, keeping the same +16 headroom, and both new Describes added to
RequiredDescribes -- load-bearing here, because a vacuously-green block is
exactly what a skip ceiling cannot distinguish from one that stopped running.

Verified under both editions (pwsh 7: 84 passed / 0 failed; WinPS 5.1: 17
passed / 0 failed / 67 skipped, container ok on both) across this file,
Build-PfbCapabilityMap.Tests.ps1 and PfbSpecTools.ContextScope.Tests.ps1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@juemerson-at-purestorage
juemerson-at-purestorage merged commit 415f05f into dmann000:main Aug 20, 2026
5 checks passed
@juemerson-at-purestorage
juemerson-at-purestorage deleted the test/issue-112-contextscope-version-tripwire branch August 25, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant