[compiler-conformance] add implementation-neutral golden fixtures - #1829
Draft
imdreamrunner wants to merge 3 commits into
Draft
[compiler-conformance] add implementation-neutral golden fixtures#1829imdreamrunner wants to merge 3 commits into
imdreamrunner wants to merge 3 commits into
Conversation
Extracted from facebook#1623 (2f2281b) without the Rust/SWC plugin, so the behavioral contract can land independently of the Rust implementation. packages/compiler-conformance holds eight fixtures as plain data: a manifest saying how to compile the entry file, and a recorded result pinning the generated JavaScript, StyleX metadata, generated CSS and the diagnostics. Nothing in it is Babel-specific, so a compiler written in any language can read the fixtures and check itself against the same contract. The Babel plugin is the first consumer, through an adapter that keeps parser options, console capture and rule processing out of the shared package. Generated JavaScript is compared as a normalized syntax tree, so how an implementation prints a program is not part of the contract. Metadata, CSS and diagnostics are compared exactly, after absolute paths are tokenized and implementation-specific diagnostic decoration is removed. define-vars-theme exercises a successful defineVars/createTheme transform. In facebook#1623 it expected a module-resolution error, which pinned a misconfiguration rather than behavior; failed transforms are still covered by validation-error.
|
@imdreamrunner is attempting to deploy a commit to the Meta Open Source Team on Vercel. A member of the Team first needs to authorize it. |
…omparing Babel's code frame syntax-highlights whenever it believes the environment supports color, and `@babel/code-frame` treats a set `CI` variable as exactly that. So on GitHub Actions the validation-error fixture's message arrived wrapped in SGR escapes, the code-frame matcher no longer recognized the excerpt lines it was meant to drop, and the whole frame was compared against a recorded message that has none. The same test passes on a developer's machine, where nothing colorizes. Color is presentation, never contract, so normalizeDiagnostic now removes escape sequences before it does anything else -- which also keeps them from hiding an absolute path from the path tokenizer.
Avoid semantic false positives around __proto__, malformed adapter results, diagnostic prefixes, code-frame detection, and path normalization. Keep fixture defaults isolated and parse identical JavaScript before declaring it equivalent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed / motivation ?
Extracted the implementation-neutral golden test suite from #1623 (commit
2f2281bcc9adae76929c2c7a32a5e03e2aa45640) so the behavioral contract can landindependently of the Rust implementation.
Nothing here adds or depends on the Rust/SWC plugin. No changes to
packages/@stylexjs/swc-plugin, no Rust configuration or dependencies, no RustCI jobs, and none of the unrelated tests from that PR.
The shared suite
packages/compiler-conformanceis a new private workspace package holding eightgolden fixtures as plain data. Each fixture is a directory with:
manifest.json— how to compile the entry (entry,syntax,pluginOptions,processOptions)expected.json— the recorded resultThe recorded result pins all four observable outputs: generated JavaScript,
StyleX metadata, generated CSS, and diagnostics. Nothing in the package is
Babel-specific, so a compiler written in any language can read the fixtures and
check itself against the same contract.
README.mddocuments the schema andwhat an adapter has to provide.
Babel as the first consumer
packages/@stylexjs/babel-plugin/test-utils/babel-conformance-adapter.jscompiles one fixture and returns the result in the shared shape. Parser
configuration, console capture and
processStylexRulesall stay here.packages/@stylexjs/babel-plugin/__tests__/golden-fixtures-test.jsruns everyfixture through that adapter.
Re-record with
STYLEX_UPDATE_GOLDEN=1 npx jest golden-fixturesfrom thebabel-plugin package.
Coverage
create-basicstylex.createdefine-vars-themedefineVars+createTheme, incl. aprefers-color-schemeoverridekeyframes-and-whenkeyframesandwhen.ancestoroptions-warningprops-and-mergepropswith merged styles, in JSXrewrite-theme-extensionrewriteAliaseswith a cross-file themeruntime-injectionruntimeInjectionvalidation-errorBetween them: generated JavaScript, metadata, CSS, warnings, errors, and both
successful and failed transforms.
How results are compared
Generated JavaScript is compared semantically, as a normalized syntax tree
(
@babel/parser; positions, comments and raw literal text stripped, keyssorted). Whitespace, quote style, semicolons, comments, how a non-computed
property key is spelled, and property shorthand (except the semantic
__proto__case) are all ignored, so no implementation has to match another'sprinter. When the programs genuinely
differ, the test falls back to asserting on the sources so the failure shows a
readable diff.
Metadata, CSS, diagnostics and status are compared exactly, after
deterministic normalization: line endings normalized, trailing whitespace
stripped from each CSS line, absolute paths replaced with
<FIXTURE_ROOT>/<REPO_ROOT>, and — in diagnostics — the location prefix and numberedsource excerpt dropped and the adapter's declared leading implementation tag
collapsed to
[stylex].Object keys are sorted; array order is preserved, because rule order is part of
the contract.
These rules are themselves tested, without involving any compiler, in
packages/compiler-conformance/__tests__/conformance-harness-test.js.define-vars-themeIn #1623 this fixture recorded a module-resolution error, which pinned a
misconfiguration rather than any behavior. It now exercises a successful
defineVars/createThemetransform: the entry is a.stylex.jsfile and thefixture sets
unstable_moduleResolution: {"type": "commonJS"}. Failed-transformcoverage is still provided by
validation-error.The canonical name
commonJSresolution derives for it —compiler-conformance:fixtures/define-vars-theme/input.stylex.js— is hashedinto the generated variables. It is stable across checkouts (verified by copying
the package elsewhere and recompiling), but changes if the fixture moves or the
package is renamed. The README says so.
Linked PR/Issues
Extracted from #1623.
Additional Context
On the recorded output. The
expected.jsonfiles are regenerated fromcurrent
mainrather than copied from #1623. The ones in that PR look stale —their recorded
jsis missing the copyright header that their owninput.jsfiles carry. Regenerating avoids importing a stale baseline; it changes nothing
about the comparison, since the AST normalizer ignores comments.
Validation, run locally on Node 24:
yarn build— passyarn test:packages— pass, including the 8 golden fixtures and the 39harness tests
yarn prettier:report— passyarn lint:report— passyarn.lockis unchanged:@babel/parser@^7.29.0resolves to an entry thelockfile already has.
Pre-flight checklist
Contribution Guidelines