Skip to content

[compiler-conformance] add implementation-neutral golden fixtures - #1829

Draft
imdreamrunner wants to merge 3 commits into
facebook:mainfrom
imdreamrunner:golden-fixtures-shared
Draft

[compiler-conformance] add implementation-neutral golden fixtures#1829
imdreamrunner wants to merge 3 commits into
facebook:mainfrom
imdreamrunner:golden-fixtures-shared

Conversation

@imdreamrunner

@imdreamrunner imdreamrunner commented Aug 28, 2026

Copy link
Copy Markdown

What changed / motivation ?

Extracted the implementation-neutral golden test suite from #1623 (commit
2f2281bcc9adae76929c2c7a32a5e03e2aa45640) so the behavioral contract can land
independently of the Rust implementation.

Nothing here adds or depends on the Rust/SWC plugin. No changes to
packages/@stylexjs/swc-plugin, no Rust configuration or dependencies, no Rust
CI jobs, and none of the unrelated tests from that PR.

The shared suite

packages/compiler-conformance is a new private workspace package holding eight
golden fixtures as plain data. Each fixture is a directory with:

  • manifest.json — how to compile the entry (entry, syntax,
    pluginOptions, processOptions)
  • the entry file, plus anything it imports
  • expected.json — the recorded result

The recorded result pins all four observable outputs: generated JavaScript,
StyleX metadata, generated CSS, and diagnostics. Nothing in the package is
Babel-specific, so a compiler written in any language can read the fixtures and
check itself against the same contract. README.md documents the schema and
what an adapter has to provide.

Babel as the first consumer

  • packages/@stylexjs/babel-plugin/test-utils/babel-conformance-adapter.js
    compiles one fixture and returns the result in the shared shape. Parser
    configuration, console capture and processStylexRules all stay here.
  • packages/@stylexjs/babel-plugin/__tests__/golden-fixtures-test.js runs every
    fixture through that adapter.

Re-record with STYLEX_UPDATE_GOLDEN=1 npx jest golden-fixtures from the
babel-plugin package.

Coverage

Fixture Covers
create-basic stylex.create
define-vars-theme defineVars + createTheme, incl. a prefers-color-scheme override
keyframes-and-when keyframes and when.ancestor
options-warning a transform that succeeds and still reports a diagnostic
props-and-merge props with merged styles, in JSX
rewrite-theme-extension rewriteAliases with a cross-file theme
runtime-injection runtimeInjection
validation-error a failed transform

Between them: generated JavaScript, metadata, CSS, warnings, errors, and both
successful and failed transforms.

How results are compared

Generated JavaScript is compared semantically, as a normalized syntax tree
(@babel/parser; positions, comments and raw literal text stripped, keys
sorted). Whitespace, quote style, semicolons, comments, how a non-computed
property key is spelled, and property shorthand (except the semantic
__proto__ case) are all ignored, so no implementation has to match another's
printer. When the programs genuinely
differ, the test falls back to asserting on the sources so the failure shows a
readable diff.

Metadata, CSS, diagnostics and status are compared exactly, after
deterministic normalization: line endings normalized, trailing whitespace
stripped from each CSS line, absolute paths replaced with <FIXTURE_ROOT> /
<REPO_ROOT>, and — in diagnostics — the location prefix and numbered
source excerpt dropped and the adapter's declared leading implementation tag
collapsed to
[stylex].
Object keys are sorted; array order is preserved, because rule order is part of
the contract.

These rules are themselves tested, without involving any compiler, in
packages/compiler-conformance/__tests__/conformance-harness-test.js.

define-vars-theme

In #1623 this fixture recorded a module-resolution error, which pinned a
misconfiguration rather than any behavior. It now exercises a successful
defineVars/createTheme transform: the entry is a .stylex.js file and the
fixture sets unstable_moduleResolution: {"type": "commonJS"}. Failed-transform
coverage is still provided by validation-error.

The canonical name commonJS resolution derives for it —
compiler-conformance:fixtures/define-vars-theme/input.stylex.js — is hashed
into the generated variables. It is stable across checkouts (verified by copying
the package elsewhere and recompiling), but changes if the fixture moves or the
package is renamed. The README says so.

Linked PR/Issues

Extracted from #1623.

Additional Context

On the recorded output. The expected.json files are regenerated from
current main rather than copied from #1623. The ones in that PR look stale —
their recorded js is missing the copyright header that their own input.js
files carry. Regenerating avoids importing a stale baseline; it changes nothing
about the comparison, since the AST normalizer ignores comments.

Validation, run locally on Node 24:

  • yarn build — pass
  • yarn test:packages — pass, including the 8 golden fixtures and the 39
    harness tests
  • yarn prettier:report — pass
  • yarn lint:report — pass

yarn.lock is unchanged: @babel/parser@^7.29.0 resolves to an entry the
lockfile already has.

Pre-flight checklist

Extracted from facebook#1623 (2f2281b) without the Rust/SWC plugin, so the
behavioral contract can land independently of the Rust implementation.

packages/compiler-conformance holds eight fixtures as plain data: a
manifest saying how to compile the entry file, and a recorded result
pinning the generated JavaScript, StyleX metadata, generated CSS and the
diagnostics. Nothing in it is Babel-specific, so a compiler written in
any language can read the fixtures and check itself against the same
contract.

The Babel plugin is the first consumer, through an adapter that keeps
parser options, console capture and rule processing out of the shared
package.

Generated JavaScript is compared as a normalized syntax tree, so how an
implementation prints a program is not part of the contract. Metadata,
CSS and diagnostics are compared exactly, after absolute paths are
tokenized and implementation-specific diagnostic decoration is removed.

define-vars-theme exercises a successful defineVars/createTheme
transform. In facebook#1623 it expected a module-resolution error, which pinned a
misconfiguration rather than behavior; failed transforms are still
covered by validation-error.
@vercel

vercel Bot commented Aug 28, 2026

Copy link
Copy Markdown

@imdreamrunner is attempting to deploy a commit to the Meta Open Source Team on Vercel.

A member of the Team first needs to authorize it.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 28, 2026
…omparing

Babel's code frame syntax-highlights whenever it believes the environment
supports color, and `@babel/code-frame` treats a set `CI` variable as
exactly that. So on GitHub Actions the validation-error fixture's message
arrived wrapped in SGR escapes, the code-frame matcher no longer
recognized the excerpt lines it was meant to drop, and the whole frame was
compared against a recorded message that has none. The same test passes on
a developer's machine, where nothing colorizes.

Color is presentation, never contract, so normalizeDiagnostic now removes
escape sequences before it does anything else -- which also keeps them
from hiding an absolute path from the path tokenizer.
Avoid semantic false positives around __proto__, malformed adapter results, diagnostic prefixes, code-frame detection, and path normalization. Keep fixture defaults isolated and parse identical JavaScript before declaring it equivalent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant