Skip to content

ESM migration: convert lib batch 3 (printing and ui) - #9069

Draft
eventualbuddha wants to merge 8 commits into
brian/esm-lib-batch-2from
brian/esm-lib-batch-3
Draft

ESM migration: convert lib batch 3 (printing and ui)#9069
eventualbuddha wants to merge 8 commits into
brian/esm-lib-batch-2from
brian/esm-lib-batch-3

Conversation

@eventualbuddha

Copy link
Copy Markdown
Contributor

Overview

Library batch 3: printing and ui — the two links that had to come next, and the ones carrying the styled-components work. printing was the only library convertible at all (everything else was still blocked), and converting it unblocks ui, which in turn unblocks backend, ballot-encoder, fujitsu-thermal-printer and monorepo-utils for the next batch.

Three commits:

  1. convert printing to ESM
  2. normalize the styled-components default import in ui — 115 modules, a no-op for the current CommonJS build, landed separately so the mechanical churn is reviewable on its own and every commit is green.
  3. convert ui to ESM — the codemod (1385 specifiers across 406 files) plus the manual work below.

libs/ui is the largest package in the repo at 476 TypeScript files, and unlike the frontend libraries it is on node resolution, because its output is loaded by node — backends render reports and ballots through it. That is what makes the interop work below necessary rather than cosmetic.

Reviewer's main focus

1. styled-components (commit 2). v5 ships CommonJS only, so under node's ESM interop import styled from 'styled-components' evaluates to module.exportsstyled.div is undefined and every template literal throws at import time. Vitest and Vite hand back styled itself instead, so a fix written for one loader breaks the other. src/styled.ts accepts either shape; its type comes from the package's own StyledInterface rather than typeof styledDefault.default, because CommonJS and ESM type resolution disagree about what a default import of a CommonJS module is and this file has to type-check on both sides of the flip.

2. Three more dependencies with the same disagreement, each handled where it is used, and each caught a different way:

dependency symptom caught by
qrcode.react import { QRCodeSVG } type-checks but fails to load — node cannot see the package's named exports; only the namespace's default has them, and Vite/vitest are the reverse importing the built output under node
i18next one types entry for both builds, so TS models the default as module.exports though node resolves the ESM build tsc
@testing-library/user-event v13 CommonJS default import; affects 58 test files tsc

3. ui's build configuration. i18next-parser.config.js becomes ESM rather than .cjs — i18next-parser looks for .js/.mjs/.json/.ts/.yaml (never .cjs) and loads it with import(). This one is load-bearing: build:app-strings-catalog runs inside build:self, so getting it wrong fails every build of ui and therefore of the repo. The existing comment explaining why the config is JavaScript rather than TypeScript still applies and now records the ESM part too.

4. The dev CLIs (three in printing, two in ui) import compiled output instead of installing esbuild-runner, which is a CommonJS require hook that can neither load ESM sources nor resolve the .js specifiers those sources now use. Four of them ran TypeScript that lived in scripts/, which the build does not compile, so there was nothing to import; those modules moved under src/ with a matching narrow coverage.exclude — they sat outside src/ before and so were never counted, and the two ui ones write back into src/, which is called out where the paths are resolved.

Also worth knowing: two specifier fixes the codemod cannot make. ./normalize.css is really normalize.css.ts, and the codemod skips anything ending in .css assuming it is a stylesheet. @testing-library/jest-dom/matchers needs its .js because that package has no exports map, so node16 resolves the subpath as a file path.

Demo Video or Screenshot

N/A — module-system change, no user-facing behavior change.

Testing Plan

Per package, locally:

printing ui
tsc --build clean clean
eslint / stylelint clean clean, incl. the app-strings-catalog check
vitest run 74/74, coverage 100%/99% 939 passed / 2 skipped over 168 files, coverage debt unchanged from baseline
built output under plain node imports all 255 modules import
CLIs all three print usage; generate-m404n-ppd reproduces the committed PPD byte for byte both run; generate-font-awesome-styles and the strings catalog regenerate identically

The whole-build/ import sweep is the check that matters most here, and it is new in this batch. The unit tests run under vitest, which resolves modules the way Vite does — the opposite side of every interop question in this PR — so they cannot see a node-only load failure. Sweeping all 255 built modules under plain node is what caught qrcode.react and the require.resolve, neither of which tsc or vitest flags.

Repo-wide on this branch: type-check passes for every dependent of both packages, and script/validate-monorepo is clean. CI covers the rest, including the frontend builds (Vite consuming ESM ui) and the integration tests that render through printing and ui inside real backends.

Checklist

  • I have prefixed my PR title with "VxDesign: ", "VxPollBook: ", or "HWTA: " if my change is specific to one of those products.
  • I have added logging where appropriate for any new user actions.
  • I have added the "user-facing-change" label to this PR, if relevant, to automate an announcement in #machine-product-updates.

🤖 Generated with Claude Code

@eventualbuddha

Copy link
Copy Markdown
Contributor Author

Parking this in draft — all 7 app-backend test suites fail, and the cause is test infrastructure rather than the libraries themselves. Recording the diagnosis so far.

Symptom. configureElectionPackageFromUsb runs the real election-package authentication and dies on Missing required VX_MACHINE_TYPE env var, because vi.mock("@votingworks/utils") — which the tests use to enable SKIP_ELECTION_PACKAGE_AUTHENTICATION — no longer reaches libs/backend.

Evidence (reproduces locally, so no CI round-trips needed):

  • Instrumenting the decision point in libs/backend/src/election_package/election_package_io.ts shows it holding the real compiled isFeatureFlagEnabled, while a probe in the test file shows the mock. So there are two instances of @votingworks/utils in one test run.
  • Checking out the commit before the ui flip (with printing already ESM) and rebuilding, the same probe shows libs/backend holding the mock. So converting ui is what splits the instances.

Fixes tried and ruled out, all at the test-config layer:

  1. Aliasing @votingworks/utils to source — utils then loads from source, still unmocked.
  2. Aliasing @votingworks/printing to source — no change.
  3. server.deps.inline: [/@votingworks\//] — no change.

That the config layer cannot fix it suggests the problem is module-system consistency: libs/backend is CommonJS and now sits between ESM (ui, printing) and CommonJS (utils), and vitest cannot keep one instance across that boundary.

Worth stressing: this is test-harness breakage, not product breakage. The converted libraries are sound — all 255 built ui modules import under plain node, a mark-frontend Vite build succeeds, and all 13 dependents type-check.

Next step is probably to extend this PR to convert libs/backend as well and re-measure, and if that is not enough, to accept that the remaining spine (backend, utils, types, basics, …) has to convert in one PR rather than in waves — partial conversion is what creates the mixed graph.

@eventualbuddha

Copy link
Copy Markdown
Contributor Author

Resolved — pushed as c106152a8d. Root cause was load ordering, not duplicate modules, and the fix touches no library code.

Tagging the module and logging from both sides showed one instance with two views:

backend sees utils instance: build:3ve7d | mocked? false
test file        utils    = build:3ve7d | mocked? true

Each test/setupTests.ts imported @votingworks/printing at module scope for one cleanupCachedBrowser() call. Setup files run before the test file, so before any vi.mock. As CommonJS printing was externalized and never entered vitest's module runner; as ESM it is processed by the runner, so importing it instantiates part of the graph pre-mock and those modules keep real bindings.

Fix: load printing inside the hook with vi.importActual (same instance when nothing is mocked, so cleanup still targets the browser the tests created), plus one narrowed node:fs stub in admin-backend's multi_station_config.test.ts, which replaced node:fs wholesale and so starved printing's import-time config read.

Measured on admin-backend, the hardest case: eager import → 31 tests failing; await import → 462 pass, 1 suite failing; vi.importActual + narrowed stub → 44/44 files, 462/462.

Two options ruled out with measurements rather than argument:

  • Aliasing internal packages to source cannot fix this — aliases address module identity, this is timing. Aliasing utils, aliasing printing, and server.deps.inline: [/@votingworks\//] all left the mock unapplied.
  • Making printing's config load lazy would have removed the hazard at its root, but SUPPORTED_PRINTER_CONFIGS feeds three exported consts with 168 usages across 45 files.

Follow-up worth considering: those same setup files also import fixtures, image-utils and test-utils at module scope, so each will reintroduce this when its batch converts. An eslint-plugin-vx rule banning module-scope @votingworks/* imports in **/setupTests.ts would catch it at authoring time.

@eventualbuddha
eventualbuddha force-pushed the brian/esm-lib-batch-3 branch from c106152 to 8960c69 Compare August 5, 2026 21:36
@eventualbuddha
eventualbuddha force-pushed the brian/esm-lib-batch-3 branch from 8960c69 to d8581a7 Compare August 5, 2026 22:11
@eventualbuddha
eventualbuddha force-pushed the brian/esm-lib-batch-3 branch 2 times, most recently from 2a62e7f to dcf7d98 Compare August 5, 2026 22:25
@eventualbuddha
eventualbuddha force-pushed the brian/esm-lib-batch-3 branch from dcf7d98 to 2255d77 Compare August 5, 2026 22:33
@eventualbuddha
eventualbuddha force-pushed the brian/esm-lib-batch-3 branch from 2255d77 to f32f12b Compare August 5, 2026 22:37
@eventualbuddha
eventualbuddha force-pushed the brian/esm-lib-batch-3 branch from f32f12b to 4324ec7 Compare August 5, 2026 22:39
@eventualbuddha
eventualbuddha force-pushed the brian/esm-lib-batch-3 branch 2 times, most recently from f38ed03 to 976bf26 Compare August 6, 2026 01:20
@eventualbuddha
eventualbuddha force-pushed the brian/esm-lib-batch-3 branch 2 times, most recently from 865d9dd to b517172 Compare August 6, 2026 18:08
eventualbuddha and others added 8 commits August 7, 2026 17:36
Codemod: `"type": "module"` + `exports`, explicit `.js` specifiers, `import.meta.dirname`.

Manual work:

- **The three `scripts/` CLIs** (`printer`, `generate-m404n-ppd`, `render-tally-report`) import the compiled output instead of installing `esbuild-runner`. That hook is a CommonJS `require` hook: it can neither load this package's ESM sources nor resolve the `.js` specifiers they now use. These are extensionless files, so node takes their module system from this package.json and was already parsing them as ESM the moment `"type": "module"` landed.

  Two of them ran TypeScript that lived in `scripts/`, which `tsconfig.build.json` does not compile, so there was nothing built to import. Those two modules move to `src/scripts/` to be compiled like everything else — with a matching `coverage.exclude`, since they sat outside `src/` before and so were never counted. The `eslint` `no-console` override for `scripts/**` extends to their new home.

- `src/render.test.tsx` took `styled` as a default import. styled-components v5 ships CommonJS only, so node's ESM interop yields `module.exports` and `styled.div` is `undefined`; vitest hands back `styled` itself. New `test/styled.ts` normalizes both shapes (same fix as `libs/hmpb/src/styled.ts` — here it is test-only, so it lives in `test/`).

- `.lintstagedrc.js` renamed to `.cjs`.

Verified: `tsc --build` and `eslint` clean; `vitest run` 74/74 and coverage still 100%/99%; `build/index.js` imports under plain `node`; all three CLIs print usage; and re-running `generate-m404n-ppd` reproduces the committed PPD byte for byte.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPUVQen9XKKg9E4JC1yaU
Preparation for converting ui to ESM, landed separately so it can be reviewed and validated on its own: this commit is a no-op for the current CommonJS build, and every check passes with or without it.

styled-components v5 ships CommonJS only. Under node's ESM interop `import styled from 'styled-components'` evaluates to `module.exports`, so `styled.div` is `undefined` and every template literal throws at import time — which matters here because ui renders server-side inside backends, not only in the browser. Vitest and Vite hand back `styled` itself instead (vitest applies its own `interopDefault`; Vite resolves the package's ESM build), so a fix written for one loader breaks the other.

New `src/styled.ts` accepts either shape, and the 115 modules that used the default import now take `styled` from there. Its type comes from the package's own `StyledInterface` rather than `typeof styledDefault.default`, because CommonJS and ESM type resolution disagree about what a default import of a CommonJS module is, and this file has to type-check under both — before the flip and after.

Named imports are untouched (`css`, `keyframes`, `ThemeProvider`, `DefaultTheme`, …): node's CommonJS named-export detection resolves those correctly, and the type-only ones never reach runtime.

Verified: `tsc --build`, `eslint` and `stylelint` clean; `vitest run` 939 passed / 2 skipped across 168 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPUVQen9XKKg9E4JC1yaU
Codemod: `"type": "module"` + `exports`, 1385 explicit `.js` specifiers and 13 `import.meta` globals across 406 files. (The styled-components interop this needed landed in the previous commit.)

Manual work, grouped by what broke:

**Build configuration.** `i18next-parser.config.js` becomes ESM. It cannot simply be renamed: i18next-parser looks for `i18next-parser.config.{js,mjs,json,ts,yaml,yml}` and would not find a `.cjs`, and it loads the config with `import()`, so `export default` is what works. Its header comment already explained why it is plain JavaScript rather than TypeScript (a `.ts` config races on a temp file during concurrent monorepo builds); that reasoning is unchanged and now records the ESM part too. This one matters more than the usual config rename — `build:app-strings-catalog` runs as part of `build:self`, so it fails *every* build of ui, and therefore of the whole repo.

`.lintstagedrc.js` and `.stylelintrc.js` renamed to `.cjs`. `.storybook/main.ts` exports with `export default` instead of `module.exports`, and its stale `@ts-expect-error` on the `vite` import is gone (the error it suppressed does not occur under ESM resolution).

**The two `scripts/` CLIs** import compiled output instead of installing `esbuild-runner`, which is a CommonJS `require` hook and can neither load ESM sources nor resolve the `.js` specifiers they now use. Their TypeScript moved to `src/scripts/` so it is compiled at all (`tsconfig.build.json` only covers `src`), with a matching `coverage.exclude` — those modules sat outside `src/` before and so were never counted. Both write files back into `src/`, so their paths are now resolved relative to the compiled location; that is called out where it happens. `build:app-strings-catalog` compiles first, so it still works when lint-staged invokes it on its own.

**Three more CommonJS dependencies whose default export ESM and bundlers disagree about**, each fixed where it is used:

- `qrcode.react` — `import { QRCodeSVG }` type-checks but fails to *load* under node ESM: node's named-export detection cannot see the package's exports, and only the namespace's `default` (i.e. `module.exports`) carries them. Vite and vitest do the reverse. `src/qrcode_react.ts` takes whichever has the component and re-exports it as `QrCodeSvg`.
- `i18next` — declares one `types` entry for both its CommonJS and ESM builds, so TypeScript models the default import as `module.exports` even though node resolves the ESM build, whose default *is* the instance.
- `@testing-library/user-event` v13 — CommonJS only, so the default import is `module.exports` under node while vitest unwraps it. `src/user_event.ts` normalizes it for the 58 test files plus `accessible_controllers/test_utils.tsx`; it lives in `src/` rather than `test/` because that test-utils module is compiled into the build and so cannot import from outside `rootDir`, which needs a narrow `import/no-extraneous-dependencies` exception.

**Specifier fixes the codemod could not make.** `./normalize.css` is a TypeScript module named `normalize.css.ts`, and the codemod skips anything ending in `.css` on the assumption it is a stylesheet — so it needed `.js` by hand. `@testing-library/jest-dom/matchers` needed `.js` too: that package has no `exports` map, so under node16 the subpath is resolved as a file path and needs its extension. `src/fonts/generate_font_awesome_styles.ts` used `require.resolve`, now `createRequire(import.meta.url).resolve`. And one test's deep `@votingworks/types/src/auth/...` import now uses the public `DippedSmartCardAuth` namespace.

Verified: `tsc --build`, `eslint` and `stylelint` clean, including the app-strings-catalog check; `vitest run` 939 passed / 2 skipped across 168 files with coverage debt unchanged from baseline; **all 255 built modules import successfully under plain `node`** (a whole-`build/` sweep, which is what caught the `qrcode.react` and `require.resolve` failures — neither is visible to `tsc` or to vitest); both CLIs run and reproduce their generated files byte for byte.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPUVQen9XKKg9E4JC1yaU
Converting printing and ui to ESM broke `vi.mock("@votingworks/utils")` in every app-backend test suite: `libs/backend` kept the real `isFeatureFlagEnabled`, so `SKIP_ELECTION_PACKAGE_AUTHENTICATION` never took effect, election-package authentication ran for real, and 7 suites failed on `Missing required VX_MACHINE_TYPE env var`.

It is a load-ordering problem, not duplicate modules. Tagging the module and logging from both sides shows one instance with two views:

    backend sees utils instance: build:3ve7d | mocked? false
    test file        utils    = build:3ve7d | mocked? true

Each `test/setupTests.ts` imported `@votingworks/printing` at module scope for a single `cleanupCachedBrowser()` call. Setup files run before the test file, hence before any `vi.mock`. While printing was CommonJS it was externalized and never entered vitest's module runner; as ESM it is processed by the runner, so importing it instantiates a chunk of the dependency graph pre-mock, and those modules keep their real bindings.

Fixed by loading printing inside the hook instead, via `vi.importActual` — which additionally resolves printing's own dependencies unmocked, and which returns the same module instance when nothing is mocked, so the browser it cleans up is still the one the tests created.

`admin-backend`'s `multi_station_config.test.ts` needed one more change: it replaced `node:fs` wholesale, and `importActual` unmocks the module named, not its dependencies — so printing's import-time read of its printer configs got the stub and threw on `"enable"`. Its `readFileSync` stub now defaults to the real implementation, which individual tests still override.

Note this is why aliasing packages to source cannot fix this class of failure — aliases address module identity, and this is timing. Aliasing `utils` to source, aliasing `printing`, and `server.deps.inline: [/@votingworks\//]` were all measured and left the mock unapplied.

Verified: full suites pass for all eight backends and bmd-ballot-fixtures — admin 462/462 across 44 files (31 were failing), print 69/69, central-scan 121/121, mark 128/128, mark-scan 161/161, scan 226/226, design 192/192, pollbook 218/218 — and `lint` is clean in all nine packages.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPUVQen9XKKg9E4JC1yaU
The failure fixed in the previous commit was silent and remote: one module-scope import in a setup file left a mock unapplied in a module nobody edited, in a different package. New `vx/no-esm-workspace-import-in-test-setup` catches it at authoring time, enabled for `setupTests.{ts,tsx}`, `test/setup.ts`, `test/setup_custom_matchers.ts` and `test/set_env_vars.ts`.

The rule reports only packages that are **currently** ESM, by resolving the imported package's `package.json` the way node would — walking up for `node_modules/<name>/package.json` — and checking `"type"`. That matters for two reasons. Today there are no violations, so this adds no churn: every module-scope workspace import left in a setup file is `fixtures`, `image-utils` or `usb-drive`, all still CommonJS, and all harmless while they are externalized rather than run inside vitest's module runner. And it means each package starts being flagged as it converts, so the fix lands in that batch's PR rather than being paid for speculatively now — there are 44 such imports across 36 setup files, some of which (registering matchers with `expect.extend`) need thought rather than a mechanical rewrite.

Type-only imports are not reported, since they are erased. Unresolvable packages and unparseable manifests are not reported either: the rule fails open rather than guessing.

Verified by re-introducing the exact regression — putting the module-scope `@votingworks/printing` import back into `apps/print/backend/test/setupTests.ts` — and confirming `eslint` reports it through the package's own config, with a message naming the `vi.importActual` replacement. Rule tests 10/10; the plugin's suite stays at 483 passing with statements and lines at 100%; `lint` clean for ui, printing and design-backend, the packages whose setup files import the most.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPUVQen9XKKg9E4JC1yaU
None of these are covered by CI, which is why they were green while broken. They installed the `esbuild-runner` hook and required TypeScript sources: that hook is CommonJS-only, and it resolves `.js` specifiers literally so it cannot find the `.ts` files those specifiers now point at. They are extensionless files, so node takes their module system from the package and had been parsing them as ESM ever since `"type": "module"` landed.

Eight are fixed. Five now import compiled output directly. Three (`copy-batch`, `copy-sheets`, `simulate-check-ins`) ran TypeScript that lived in `scripts/`, which `tsconfig.build.json` does not compile, so there was nothing built to import — those modules move to `src/scripts/`, with narrow `coverage.exclude` entries because they sat outside `src/` before and so were never counted, and the `no-console` eslint override for `scripts/**` extends to their new home.

Two are deleted rather than ported: `central-scan/backend/bin/{read-qrcode,render-pages}` required `../src/cli/`, which was removed in 8a41def long before this migration, and nothing references them.

VxDesign's two CLIs get the same treatment in its own PR, alongside the startup fix, so that design's conversion is self-contained.

Verified: each of the eight runs and prints its usage or reaches a real argument or environment error; `lint` and `tsc` clean in all five packages; and full suites with coverage pass — central-scan 121/121, plus pollbook, scan, custom-paper-handler and fixture-generators.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPUVQen9XKKg9E4JC1yaU
The `bin/` commands now run compiled output rather than transpiling `src` on the fly, so following the README on a fresh checkout fails with `ERR_MODULE_NOT_FOUND` until the package is built.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPUVQen9XKKg9E4JC1yaU
Same as the previous batches: #9079 landed the convention and a
validate-monorepo check that enforces it. Produced by re-running the codemod.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPUVQen9XKKg9E4JC1yaU
@eventualbuddha
eventualbuddha force-pushed the brian/esm-lib-batch-3 branch from b517172 to 4b52e77 Compare August 8, 2026 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant