loader: standalone tsx-style npm loader package - #810
Conversation
…n plumbing
Adds runtime/loader-*.{mjs,cjs}: slim entrypoints that arm only the shared
resolve+transpile hooks (transform-core + preload-common) for
node --import <pkg> / --require <pkg> / <pkg>/esm, with the addon resolved from
a per-platform npm package via __NUB_ADDON_PATH (probed last by transform-core).
Adds npm/loader + npm/loader-<platform> package manifests, the staging script,
and version/lockstep/gitignore plumbing.
release.yml stages a loader-<platform> addon artifact per build leg and publishes the 8 addon packages then the root loader package (stable + canary), mirroring the CLI's ordering. Adds site/content/docs/loader.mdx and a tarball-install matrix under tests/loader/ (--import on every tier, --require where require(esm) exists, optional tsx differential).
|
Your Claude subscription has hit its usage limit. It resets at 12:40am (UTC). Re-trigger Pullfrog after the reset, or add an Add repo secret → · Model settings → · Setup docs → · Ask in Discord →
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Your Claude subscription has hit its usage limit. It resets at 12:40am (UTC). Re-trigger Pullfrog after the reset, or add an Add repo secret → · Model settings → · Setup docs → · Ask in Discord →
|
npm's OIDC trusted publishing cannot create a package (npm/cli#8544), so the nine new loader packages must exist and be trusted before the release that ships them. Maintainer-run, idempotent.
npm's similarity guard refuses new unscoped names near existing ones (nubjs → 403 vs fbjs/runjs); scoped names bypass the guard and the org owns the scope.
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
One correctness defect in the compat-tier arming path, plus two release-process risks worth a decision before the first tag.
Reviewed changes
- Traced the
arm()tier decision inruntime/loader-entry.mjsacross (Node version ×--import/--require× foreign loader × worker thread), againstruntime/preload.cjs's fast-tier reference. - Verified the
RUNTIME_FILEStransitive closure inscripts/build-loader-npm.mjs— every relative import of every staged file is present, andruntime/version.mjsgenuinely is not needed. - Checked the release wiring end to end: per-target addon staging,
VERSIONavailability in the two new publish steps, and version lockstep at0.8.0acrossnpm/nub,npm/loader, and all eightnpm/loader-*manifests. - Confirmed the
npm trustCLI contract and the npm/cli#8544 first-publish premise behindscripts/npm-first-publish-loader.sh.
Cleared on inspection, recording so they don't get re-raised: the root .gitignore cannot strip staged files from the tarball (npm's files[] allowlist wins); armed really is shared across a combined --import + --require delivery, because require(esm) and import() of the same resolved .mjs share one module record; the unscoped nubjs name is a deliberate carve-out and is documented as one in the wiki/agents.md diff.
⚠️ The release can leave npm half-published
The loader publish step is appended to the end of publish-npm (.github/workflows/release.yml), after @nubjs/nub and its platform packages have already published irreversibly. Nothing in the workflow gates the release on scripts/npm-first-publish-loader.sh having been run.
If it hasn't been (or its trust step failed quietly — see the inline comment on it), the step fails on the first loader package, and the tag lands with @nubjs/nub@X on npm and no matching nubjs@X. test-install blocks on publish-npm, so it won't catch it either.
Recovery is real — every step is idempotent, so running the one-time script and re-running the job fixes it. But the window where the two package families disagree is user-visible, and the ordering makes the loader the thing that fails last rather than first. Two options, both cheap: move a "loader packages exist and are trusted" precondition check into the verify job so a missing setup fails before anything publishes, or move the loader publish ahead of @nubjs/nub so the irreversible step is last.
ℹ️ tests/loader/ isn't wired into CI
tests/loader/run-matrix.sh is the only thing in the repo that packs the tarballs and installs them into a clean directory, and tests/loader/README.md says plainly that install-from-tarball is what catches addon-resolution bugs — "one shipped that way in the first cut." But nothing under .github/workflows/ references tests/loader or run-matrix, so the first real staging of these packages happens during a release.
The compat-tier defect flagged inline is exactly the shape this matrix would catch, and it needs a Node in the 18.19–22.14 band to surface. Worth at least a Linux leg on PRs touching runtime/** or npm/loader*/**.
ℹ️ Nitpicks
scripts/build-loader-npm.mjs— the comment says the manifest cross-check catches drift "or vice versa", butRUNTIME_FILES.filter((f) => !manifest.files.includes(f))only runs one direction. The reverse case (an entry dropped fromRUNTIME_FILESbut still listed infiles[]) is the one that silently produces a broken tarball. A naive reverse check would false-positive onREADME.md/LICENSE, so correcting the comment is probably the cleaner fix.site/content/docs/loader.mdxdocuments the.yaml/.toml/.json5/.jsonc/.txtimports andWorker, which is exactly the setsite/content/docs/runtime/loaders.mdxfronts with<TypesSetup>. The loader page omits it, so a reader following it gets type errors on the imports it just showed them.
Claude Opus ｜ 𝕏
…, addon-path precedence, publish hardening - The compat tier's loader worker imports its own transform-core instance, so the entry's CLOBBER_MAP.clear() never reached the map its resolveSpec reads: a user's real @js-temporal/polyfill silently bound undefined on 18.19-22.14 (verified on 20.19). The clear now rides module.register's data payload into initialize(); the CLI registers with no data and keeps its clobbers. - ensureAddonEnv no longer trusts an inherited __NUB_ADDON_PATH over its own platform-package resolution (version-skew hazard for a nested loader). - npm-first-publish-loader.sh: npm >= 11.10 floor, and the trust step verifies via npm trust list instead of swallowing failures. - release.yml: loader packages publish FIRST (before @nubjs/nub's irreversible publishes) and both publish jobs preflight that the nine packages exist. - build-loader-npm.mjs: file: URL for the dynamic import (Windows). - tests/loader: clobber.ts differential fixture (the defect above).
There was a problem hiding this comment.
ℹ️ All four inline findings from the previous review are fixed; I verified each against the code rather than the commit message and retired the threads. Two diagnosability nitpicks inline, nothing blocking.
Reviewed changes
- Re-derived the compat-tier clobber-map fix end to end:
registerLoaderWorker(runtime/preload-common.cjs:370) forwardsoptionsuntouched intomodule.register, both the ESM entry and theERR_REQUIRE_ESMfallback send{ standaloneLoader: true }, and the CLI's two call sites still pass nothing so their clobbers stay intact. The newclobber.tsfixture is non-vacuous — without the worker-side clear the compat tier printsundefined, notnumber. - Checked the
initialize(data)ordering claim against Node's source rather than the comment it rests on.module.register()blocks the calling thread onAtomics.waituntil the worker'sHooks#registerhas awaitedinitialize, so the clear lands before the firstresolve. Confirmed at thev18.19.0tag, where the 3-argdataform is also documentedadded: v18.19.0— present at the support floor, not a later addition. - Traced the reworked
ensureAddonEnvprecedence across the realm matrix (fast tier, loader worker, userworker_threads,--requirefallback, nested CLI in both directions). A worker'sprocess.envis a copy taken at creation, andloader-addon-env.mjsis the first static import ofloader-entry.mjs, so the var is always set before any worker exists — no race on either tier. - Walked the release reorder: preflight → loader platform packages → loader root →
@nubjs/nub→ types, mirrored in the canary job. The two copies are in sync,if ! npm viewis correctly exempt from the runner's defaultbash -eso the loop accumulates all nine names instead of aborting on the first miss, andnpm/nubcarries no dependency on the loader family — so the new "loader publishes,@nubjs/nubfails" shape is a re-runnable partial publish rather than a version skew. Materially better than the previous ordering. - Confirmed the rename to
@nubjs/loaderis complete: no unscopednubjspackage reference survives anywhere in the tree,run-matrix.sh's tarball-name derivation still yields the right filename for a scoped name, andmake version-checkis green at0.8.1across all eleven manifests.
Claude Opus ｜ 𝕏
… the harness README The last-resort branch that binds a parent process's __NUB_ADDON_PATH is the one path that can pair version-skewed addon and JS, so it now says so on stderr instead of succeeding silently.
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
d1ab23f is the only commit since the last review, and it closes both nitpicks from it. I verified each against the code rather than the commit message, and retired both threads.
- Warned on the inherited-addon fallback —
runtime/loader-platform.cjs:92now writes a stderr line naming the inherited path before returningtrue, so the one branch that produces a wrong-but-working addon no longer leaves zero trace. - Listed
clobber.tsin the harness fixture table —tests/loader/README.md:21, the fixture whose purpose was least guessable from its filename.
Two things I checked rather than assumed about the new warning. It cannot fire spuriously: __NUB_ADDON_PATH is written in exactly one place in the tree (loader-platform.cjs:89) and the Nub CLI never sets it, so an inherited value always originates from another standalone-loader process — precisely the version-skew case the message names. And it cannot double-print on the compat tier: the loader worker imports preload-async-hooks.mjs directly and never reaches ensureAddonEnv, so the only repetition is one line per user worker thread that re-evaluates the entry, which is the same shape the total-failure branch below it already had.
Worth noting for future runs: run_one in tests/loader/run-matrix.sh captures 2>&1, so either warning firing during the matrix surfaces as a fixture mismatch rather than passing quietly.
Claude Opus ｜ 𝕏
|
Shipped in v0.8.2: https://github.com/nubjs/nub/releases/tag/v0.8.2 |
Ships the runtime's resolve + transpile hooks as a standalone npm package:
node --import @nubjs/loader,--require @nubjs/loader,@nubjs/loader/esm. Slim entries underruntime/loader-*arm only the sharedtransform-core/preload-commonhooks; the addon rides 8@nubjs/loader-<platform>packages resolved at runtime (__NUB_ADDON_PATH, probed last). Version/lockstep/release wiring, a docs page, and a tarball-install matrix undertests/loader/are included.Name:
@nubjs/loader(barenubjsis refused by npm's similarity guard; scoped names bypass it). Before the first release that ships it, runscripts/npm-first-publish-loader.shonce — npm OIDC cannot create a package (npm/cli#8544), so the nine new packages must exist and be trusted first.Verified from a clean tarball install:
--importgreen on Node 18.19–26.7;--requireon 20.19+/22.12+.