test(COMPASS-28): move to @stoplight/spectral-core

[timdawborn]

Ticket

COMPASS-28 — Bump spot's NodeJS minimum from 18 to 22

PR 7 of 9. Staying in draft until the whole chain is verified.

Stacked on #2723 → #2722 → #2721 → #2720 → #2719. This PR's own commit is 5b783e6.

What

@stoplight/spectral@5.9.2 is deprecated and was superseded by a package split, not a version bump. Replaced with @stoplight/spectral-core, -parsers and -rulesets — all dev-only, all absent from the image (verified). Supersedes Dependabot #1987.

Three specs used spectral, not one — OpenAPI 3, OpenAPI 2, and JSON Schema. I found the second by grep and the third only when its suite failed, which is a good argument for grepping the package name before starting rather than after.

Four rules in each ruleset had been renamed upstream

A name-for-name port would have looked clean and silently changed what is checked. setRuleset throws on an unknown rule name, so these surfaced loudly — but only once the names were being resolved at all:

v5	v6
operation-2xx-response	operation-success-response
example-value-or-externalValue	oas3-examples-value-or-externalValue
oas3-unused-components-schema	oas3-unused-component
oas3-valid-example	oas3-valid-media-example + oas3-valid-schema-example
oas2-valid-example	oas2-valid-media-example + oas2-valid-schema-example

example-value-or-externalValue is dropped from the OpenAPI 2 ruleset rather than renamed: v6 only has an oas3- form, which would be skipped against an OpenAPI 2 document anyway.

A real bug this surfaced — deferred to COMPASS-29

The schema half of the split valid-example rules is left off. Enabling it is a coverage increase rather than a port, and it reports six real findings against the schemaprops contract:

schema is invalid: data must have property minimum when property exclusiveMinimum is present   (x3)
schema is invalid: data must have property maximum when property exclusiveMaximum is present   (x2)
"default" property must match format "date-time"                                              (x1)

The generator emits, for example:

{ "type": "number", "format": "float", "maximum": 99.95, "exclusiveMinimum": false, ... }

exclusiveMinimum with no minimum. Under OAS 3.0 that keyword is a boolean modifier on minimum, so the schema is invalid against the meta-schema — and false is the default, so emitting it at all is redundant. Confirmed spectral v5's single rule never checked this, so nothing regressed; the rule was simply never on.

Fixing the generator changes generated output and needs its own snapshot review, so it is COMPASS-29, linked from the deferral comment at each site.

Each ruleset now has a test that proves it is doing something

Every other spectral assertion in these specs is toHaveLength(0) — which a ruleset that enables nothing satisfies exactly as well as a clean document. With 20-odd rules per ruleset and four of them renamed in this very bump, that is a lot of silent inertness to leave unguarded.

So each spec lints a document that breaks specific enabled rules and asserts the codes it gets back. Mutation-checked:

Mutation	Result
rules list emptied	new test fails, other 23 still pass
an enabled rule renamed to a bogus name	setRuleset throws (assertExistingRule)

I initially justified this test as guarding against a format mismatch, and also declared formats on each ruleset for the same reason. Both were wrong and are gone: the rules inherited from oas carry their own formats, which is what decides application (a ruleset-level declaration changed nothing in an A/B/C comparison), and a document spectral cannot place reports unrecognized-format rather than passing silently. The comments now state only what the tests actually pin.

The JSON Schema spec's spectral call could not fail

It registered the draft-07 format and never loaded a ruleset, so run executed with zero rules:

findings on a valid doc  : 0
findings on garbage      : 0
rules registered         : 0

The assertion held regardless of the document. v6 cannot reproduce that setup (it requires a ruleset), and there is no off-the-shelf JSON Schema ruleset to port to — so rather than delete a dead assertion, it now validates against the draft-07 meta-schema via ajv, already a dependency. Mutation-checked: feeding it required: 42 fails with must be array.

Supply chain

@scarf/scarf arrives through every spectral v6 package. Its postinstall is node ./report.js — install analytics, "like Google Analytics for your npm packages" — and it carries no functionality. strictDepBuilds caught it and its build is denied, alongside core-js and unrs-resolver.

@stoplight/spectral-formats was declared and then became unused once the inert formats declarations came out; removed rather than left in the manifest.

How this was verified

Check	Result
pnpm build	0
pnpm test	0 — 54 suites, 557 tests (2 new guards), 44 snapshots
pnpm lint:check	0
pnpm build-docs	0
docker build + check-image-parity	0 — all checks passed
@stoplight/* in the image	absent (dev-only)

[timdawborn]

Automated review — /pr-review-toolkit:review-pr

Agents run: code-reviewer, pr-test-analyzer. Scoped to this PR's own commit (5b783e6). Suite confirmed green on this commit: 10 suites, 258 tests, 44 snapshots.

The sharp finding: this PR replaced a vacuous assertion with a weak one, and the mutation check I ran happened to pick the single case the weak check catches.

🔴 Critical (2)

1. json-schema.spec.ts:16-21 — validateSchema is the weakest ajv check available, and misses a live bug class. Reproduced against the installed ajv 8.18.0:

case	validateSchema	compile
dangling $ref	true	THROWS
typo'd keyword (minimun)	true	accepts
unknown format	true	accepts
required: 42	false	THROWS

The generator emits $ref: "#/definitions/User" (snapshot line 61), so dangling refs are a live bug class this assertion cannot see. Add validator.compile(schema) alongside — it throws on an unresolvable $ref at any strict setting.

2. Same file — the spec that had the vacuous assertion is the only one that got no guard. assertValidJsonSchema is only ever called on valid generator output (:34, :48, :60); nothing proves it can fail. The two oas specs each received exactly this mutation check. Add one negative case (type: "strng" → expect false).

Worth being explicit about why this matters: my mutation check for the replacement used required: 42 — the one row above where validateSchema returns false. That gave false confidence that the new assertion was strong.

🟡 Important (2)

3. json-schema.spec.ts:16 — strict: false is inert. Strict-mode checks fire at compile(), never inside validateSchema; verified identical results for strict true/false across all 7 cases. It doesn't weaken anything today, but it is dead config that will silently disarm a later compile call. Drop it, or keep it with a comment naming what it suppresses.

4. openapi2.spec.ts:61 / openapi3.spec.ts:79 — the guard test overclaims. I verified: flipping oas2-schema, typed-enum and operation-success-response to false leaves the oas2 guard green (20/20 pass). It exercises 2 of 19 rules there, and neither is oas2--prefixed, so no format-specific rule is covered anywhere in that file. The name ("the enabled rules are applied") claims more than it enforces — rename it to the wiring property it actually pins.

That's the third round on this same comment in this PR, which is its own signal.

🔵 Suggestions (1)

openapi2.spec.ts, every toHaveLength(0) site — the unrecognized-format safety net documented at openapi3.spec.ts:22-26 is genuine, but it misses cross-version drift: an OpenAPI 3 document linted under the oas2 ruleset returns [], because the extended oas ruleset registers oas3 formats too. swagger: "2.0" appears only in the new fixture, never as an assertion on generator output. expect(result.swagger).toEqual("2.0") inside lint closes it.

✅ Verified sound

• Exact-code arrays are not brittle — extends: [[oas, "off"]] lands upstream additions disabled, so they cannot add a finding. Only a rename (which makes setRuleset throw) or a behaviour change in an already-enabled rule breaks the array, and both merit a failure. Keep toEqual.
• The guard does not duplicate setRuleset throwing — that validates rule names, not enablement. The reasoning in the comment checks out.
• The four renames are documented at the site with the old name; the two deliberately-off rules carry a navigable COMPASS-29 link rather than a bare "deferred"; the ruleset objects sit next to the assertions that depend on them.

Recommended action

1. Add compile() and a negative case to json-schema.spec.ts — findings 1 and 2 are the same omission and fix together.
2. Drop the inert strict: false.
3. Rename the guard test to what it enforces, or widen it to cover a format-specific rule.

[timdawborn]

Automated review, part 2 — code-reviewer

The code-reviewer agent finished after my comment above and found something that corrects both that comment and this PR's description. I verified it directly against the unpacked @stoplight/spectral@5.9.2 ruleset rather than take it on trust.

🔴 Critical — the rename history is wrong, and it is the premise for the deferral

openapi3.spec.ts:36,51 and openapi2.spec.ts:37,45.

Checked @stoplight/spectral@5.9.2/rulesets/oas/index.json for every name involved:

name	in spectral 5.9.2?
example-value-or-externalValue	absent
oas3-valid-example	absent
oas2-valid-example	absent
oas3-valid-schema-example	present — same name, all along
operation-2xx-response	present
oas3-unused-components-schema	present

So three of the names the old YAMLs enabled did not exist, and v5's loadRuleset silently ignored unknown names. Those rule lines were dead and no example validation was ever running.

Two things follow, and both contradict what I wrote:

1. oas3-valid-example was never "split into" the media and schema halves. oas3-valid-schema-example existed in 5.9.2 under exactly that name; it simply was never enabled. My comment at openapi3.spec.ts:51 — "the media half is what v5's single rule actually checked" — is false, and so is the split row in the description's rename table.
2. oas3-valid-media-example and oas2-valid-media-example are coverage increases, not ports — the same category I used to justify deferring the schema half.

The code stays as-is; the narrative needs fixing, in the two comments and the description. It matters because the COMPASS-29 owner would otherwise inherit "v6 got stricter" when the truth is "this rule was never on" — a different investigation. I have corrected COMPASS-29's description accordingly.

🟡 Important (3)

setRuleset(ruleset as any) is avoidable — openapi3.spec.ts:68-69, openapi2.spec.ts:54-55. const ruleset: RulesetDefinition = { extends: [[oas as RulesetDefinition, "off"]], … } compiles clean under the repo's tsc --strict and drops the eslint-disable. as any currently silences the whole literal; the typed form catches a mistyped "off" — precisely the "extends/rules wiring changed" failure the new canary exists to catch.

pnpm-workspace.yaml:22 — verifiable-claim error. The comment says "all four @stoplight/spectral-* packages"; the tree has seven (-formats, -functions, -ref-resolver, -runtime arrive transitively), four of which pull @scarf/scarf. Drop the count.

Change-narration comments — json-schema.spec.ts:10-16, openapi3.spec.ts:73, openapi2.spec.ts:22. "Renamed from…", "This replaces a spectral call that could not fail", "Spectral 6 lints a parsed document rather than…" are provenance and change-narration; the commit message already carries them verbatim.

✅ Verified sound

• The remapping itself is faithful — all 20 openapi3 and 19 openapi2 names exist in the installed 1.22.7, no dead names remain, and setRuleset would throw on one. Behaviourally the new ruleset is a strict superset of what actually ran under v5.
• Dropping example-value-or-externalValue from the OAS2 ruleset loses nothing, twice over: the v6 rule is formats: OpenAPI 3.x only, and the old line was already inert.
• Deferring valid-schema-example is the right call — COMPASS-29 is a distinct, navigable follow-up, and fixing a generator bug inside a dependency migration would break atomicity.

Recommended action

1. Correct the two rename comments and the description's table — the code is right, the story isn't.
2. Type the ruleset and drop as any.
3. Fix the "four packages" count.

[timdawborn]

Automated review, round 2 — /pr-review-toolkit:review-pr

Agents: code-reviewer, pr-test-analyzer. Scoped to the revised commit 4735738.

The two agents disagreed on one point; I resolved it by probe and report both below.

🔴 Critical (1)

json-schema.spec.ts — adding compile() introduced a latent failure the fixtures happen to hide.

The generator emits format: "date" and format: "date-time" (json-schema-type-util.ts:49,51). No format is registered on the validator, and compile() rejects an unknown format. Verified:

validator	schema with format: "date-time"
new Ajv({ strict: true })	THROWS unknown format "date-time"
new Ajv({})	THROWS (this is strictSchema, on by default)
new Ajv({ strict: false })	compiles
addFormats(new Ajv({ strict: true }))	compiles

It ships green only because no JSON Schema fixture has a date property. The first contributor to add one gets a confusing failure in an assertion that looks unrelated. ajv-formats@^3.0.1 is already a production dependency — add addFormats(validator).

🟡 Important (3)

1. Two of the three assertions are redundant, and the docstring says the opposite. compile() runs validateSchema internally (ajv's validateSchema: true default) and throws schema is invalid: …. Confirmed with {type: "nummber"}: validateSchema returns false and compile throws. So :18-19 catch nothing :20 misses — they only improve the message. My docstring "Two assertions, because they catch different things" is inaccurate. And the errors check at :19 is separately vacuous: errors is null on success, and on failure the line above already aborts.

2. strict: true — the agents split, and both are partly right. The test-analyzer mutated strict: true → false and the minimun case failed, so it is pinned. The code-reviewer noted minimun also throws under new Ajv({}), because strictSchema is on by default. My probe confirms both: {} throws, {strict: false} does not.

So the honest reading is: the test guards against someone setting strict: false; it does not show strict: true adds anything over the default. What strict: true actually adds is strictTypes/strictTuples escalation and strictRequired. Either pin it with a shape it genuinely governs (a required naming a property absent from properties), or drop it and rewrite the comment.

3. The guard test still overclaims, after being renamed once. "…leaves the named rules enabled" reads as the whole list above it. It pins 2 of 18 enabled oas2 rules and 3 of 19 oas3 rules — and a mutation flipping five other named rules off leaves all 221 tests green. Its true scope is "not every rule is off". Name it for the codes it asserts.

🔵 Suggestions (3)

• openapi3.spec.ts:25-27 is wrong as written. unrecognized-format fires only for a document spectral can place nowhere (confirmed with {hello:"world"}). A wrong-version OpenAPI document is placeable and does pass vacuously. Narrow to "a document of neither OpenAPI version".
• Bare .toThrow() at :33 and :45 passes if any of the three internal expects throws, so it does not pin which mechanism rejected. Use .toThrow(/can't resolve reference/) and .toThrow(/unknown keyword: "minimun"/).
• Two words on the as RulesetDefinition cast. It bridges a genuine upstream imprecision — oas.rules['duplicated-entry-in-enum'].severity is widened to string in spectral-rulesets' emitted .d.ts (missing as const) — so it is correct and materially better than as any. But because it covers the whole oas object it will also swallow a future real incompatibility. Naming the widened field tells the next reader what the cast is for.
• Three positive tests are still async with nothing awaited, left over from the spectral call.

✅ Verified sound

• The corrected rename narrative checks out, independently re-verified against @stoplight/spectral@5.9.2's oas/index.json: example-value-or-externalValue, oas2-valid-example, oas3-valid-example absent; oas3-valid-schema-example, operation-2xx-response, oas3-unused-components-schema present. Every claim at the call sites and in the commit message holds, including "spectral 6 has only an oas3- form".
• compile() is load-bearing — dropping it fails both negative cases.
• No $id collision is possible — a fresh validator per call, and the generated schemas carry no $id at all, so even a shared instance would key on the object.
• strict: true does not change what the three existing generator assertions accept — they pass because the fixtures emit only type/properties/required/items/allOf/$ref, required is always derived from properties, and no tuple items exist.
• Cross-version drift is covered after all — the round-1 suggestion to assert swagger: "2.0" is redundant: all 18 oas2 lint sites sit behind a full-document snapshot whose stored text carries the version, so drift breaks 18 snapshots first.
• @scarf/scarf denial correct; COMPASS-29 is a distinct navigable follow-up; no surviving reference to the two deleted YAML rulesets in tracked files, dotfiles included.

Recommended action

1. Add addFormats(validator) — this is a real trap for the next contributor.
2. Collapse the three assertions to what compile() actually needs and fix the docstring.
3. Decide strict: pin it properly or drop it.
4. Rename the guard for the third time, to what it pins.