Skip to content

feat(node): support Alpine/musl and make the native loader libc-aware with actionable errors - #234

Open
WiktorStarczewski wants to merge 3 commits into
mainfrom
wiktor/node-musl-and-loader-diagnostics
Open

feat(node): support Alpine/musl and make the native loader libc-aware with actionable errors#234
WiktorStarczewski wants to merge 3 commits into
mainfrom
wiktor/node-musl-and-loader-diagnostics

Conversation

@WiktorStarczewski

Copy link
Copy Markdown
Collaborator

Problem

Deploying the Node.js SDK on Alpine/musl Linux (a common ECS/Docker base image) fails at import with a generic, misleading error:

Error: Miden napi module not found.
Build it with:
  cargo build -p miden-client-web --no-default-features --features nodejs --release
Or set MIDEN_MODULE_PATH to the .node file location.

Three separate defects converge into that one unhelpful message:

  1. No musl binary is published. Only node-{darwin-arm64,darwin-x64,linux-x64-gnu} ship. The gnu .node is glibc-linked (needs ld-linux-x86-64.so.2 + GLIBC_2.34) and cannot dlopen on Alpine/musl.
  2. The gnu package has no libc guard, so npm/pnpm happily install it on Alpine, where it then fails to load at runtime.
  3. loader.js swallowed the real error in a bare catch {}, so musl mismatch, a skipped optional dependency, an old glibc, and Graviton/arm64 all collapse into the same "build it with cargo" text — advice that's wrong for every one of them.

What this PR does

1. Publish a Linux musl (Alpine) binary — @miden-sdk/node-linux-x64-musl.
publish-web-sdk.yml gains a matrix leg that cross-builds the musl .node (dynamically linked; +crt-static disabled so a musl-linked Node dlopens it against its own libc), and the new platform-package dir. The leg is fully isolated so it can never regress the existing release:

  • fail-fast: false + job-level continue-on-error on the musl leg;
  • a pre-publish smoke test that require()s the built binary inside node:20-alpine — a binary that doesn't load fails the (isolated) leg, so it is not uploaded and not published, instead of shipping broken;
  • the publish loop skips any missing artifact and treats the musl publish as best-effort (a failed musl publish defers musl to the next release rather than wedging a release whose stable packages already published).

2. libc-aware loader + libc field.
getPlatformPackageName() now resolves linux-x64 to -gnu or -musl by the runtime C library (process.report glibc signal, with a shared-object rescue for atypical glibc builds). The linux packages declare "libc": ["glibc"] / ["musl"] so libc-aware managers install only the matching one. On managers that ignore libc (older npm/yarn-classic) there is no regression — glibc hosts still install -gnu via os/cpu, and the loader-side selection + better error still cover musl users.

3. Stop swallowing the error.
Failed resolution now throws an actionable diagnostic — platform/arch/libc/node, the real failure of each attempt, and targeted fixes (skipped optional dep, Alpine/musl vs glibc base image, MIDEN_MODULE_PATH). An explicit MIDEN_MODULE_PATH is now authoritative (fails fast rather than silently falling back to a different installed binary).

Example — the Alpine case now reads:

Miden napi module not found for linux-x64 (musl), Node v20.19.6.

Resolution attempts:
  - require("@miden-sdk/node-linux-x64-musl") -> MODULE_NOT_FOUND: Cannot find module '@miden-sdk/node-linux-x64-musl'

Expected the optional dependency "@miden-sdk/node-linux-x64-musl" to be installed and loadable. Common causes:
  - The optional dependency was skipped at install time (npm's cross-platform lockfile bug, ...).
  - The base image's libc or CPU does not match a published binary (e.g. an Alpine/musl or arm64 image).

Fixes:
  - Reinstall with optional dependencies on the target platform ...
  - On Alpine/musl, ensure @miden-sdk/node-linux-x64-musl is installed, or switch to a glibc base image such as node:22-bookworm-slim.
  - Or set MIDEN_MODULE_PATH to a prebuilt .node file.
  - Or build from source: ...

⚠️ Action required before the first musl release

@miden-sdk/node-linux-x64-musl is a brand-new package name and this workflow uses OIDC trusted publishing with no token fallback. npm trusted publishing is configured per package, so the first publish will fail unless the trusted publisher for this exact name is pre-registered on npmjs.com (or the first publish is done once manually with a token). The best-effort publish means a first-release miss is non-fatal (it just warns and defers musl), but pre-registering avoids a wasted cycle.

Verification

  • Loader exercised against the real module under faked os.platform/os.arch/process.report across 6 scenarios: linux+glibc→-gnu, linux+musl→-musl, atypical-glibc-without-glibcVersionRuntime→rescued to -gnu, linux-arm64→"unsupported" message, macOS→-darwin with no bogus libc label, and bogus MIDEN_MODULE_PATH→fail-fast. Prettier (3.8.1) clean; eslint camelcase satisfied.
  • Workflow YAML validated; matrix/expression semantics reviewed; publish-loop isolation reviewed.
  • Not validated locally: the actual musl cross-build (macOS host; the dep tree includes ring + libsqlite3-sys, which make musl cross-compilation the finicky part). This is deliberately guarded by the in-CI node:alpine smoke test above — a bad binary fails its isolated leg instead of publishing. If the plain cross-build proves unreliable, the fallback is to build the musl leg inside an Alpine container / the napi-rs musl image; the rest of the PR is unaffected.

Reviewed internally (loader/packaging correctness + release-pipeline regression safety); findings applied (best-effort musl publish, pre-publish Alpine smoke test, authoritative MIDEN_MODULE_PATH, libc-detection rescue, complete per-attempt diagnostics).

Follow-up (not in this PR): a committed vitest suite for the loader's pure functions (couldn't run the web-client test harness in this environment; the manual smoke test above stands in for now).

… with actionable errors

Node deployments on Alpine/musl Linux (a common ECS/Docker base image) failed
with a generic "Miden napi module not found": no musl binary was published,
the glibc package had no `libc` guard so npm installed it on Alpine where it
can't dlopen, and the loader swallowed the real require/dlopen error so every
failure collapsed to the same unhelpful message.

- Publish a new @miden-sdk/node-linux-x64-musl native binary. The publish
  workflow cross-builds it (dynamically linked, -crt-static off) and verifies
  it actually loads inside node:alpine before publishing. The musl leg is
  isolated (fail-fast: false + continue-on-error + best-effort publish) so a
  cross-build/publish hiccup never blocks the glibc/darwin binaries or WASM.
- loader.js picks the linux platform package by runtime C library (glibc vs
  musl), and on failure throws an actionable error: platform/arch/libc/node,
  the real failure of each resolution attempt, and targeted fixes. An explicit
  MIDEN_MODULE_PATH is now authoritative (fails fast instead of silently
  falling back).
- Add the `libc` field to the linux platform packages so npm/pnpm install only
  the matching binary (glibc host -> -gnu, musl host -> -musl).
@WiktorStarczewski
WiktorStarczewski requested a review from igamigo July 9, 2026 18:13

@igamigo igamigo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not super familiar with the setup, but I think there's a decent amount of room for hardening the CI and publish jobs. For example, more tests, better reports of publishing steps, failing the whole thing instead of swallowing errors, etc. I also wonder if there's more platforms we want to support.

Comment on lines +118 to +122
# The musl binary is newer/best-effort: a cross-build hiccup should leave
# this leg green so the publish job still ships the glibc + darwin
# binaries and the WASM package. The publish loop skips any missing
# artifact, so a failed musl leg simply defers musl to the next release.
continue-on-error: ${{ matrix.rust-target == 'x86_64-unknown-linux-musl' }}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think the musl build should remain best-effort once the release advertises Alpine support. Right now, the release can still pass while silently omitting the musl artifact.

Could we bootstrap the npm package first and then make this matrix leg required?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I second this. Before, the Node SDK used to be experimental so we didn't want to interfere with the Web SDK, but now it'd make sense not to publish anything if any step fails, so that publishing is all-or-nothing.

@igamigo
igamigo requested a review from JereSalo July 13, 2026 15:23

@JereSalo JereSalo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reproduced the musl build on an Ubuntu 24.04 server of mine (same image as CI). It built successfully, but the resulting .node was linked against glibc (libc.so.6 / ld-linux-x86-64.so.2), not musl, so it loaded on glibc and failed on Alpine. The cause is that the workflow sets CC=musl-gcc but not rustc's linker, so the final link still used the host's glibc compiler. After switching the step to a GNU musl-cross toolchain (rust-musl-cross), the real crate (ring + blake3 + sqlite) cross-built into a proper musl binary that loads and initializes on node:20-alpine. It's just a one-file change to the workflow, which I pushed to the branch jere/musl-cross-toolchain-fix in case you want to take it. (cargo-zigbuild was the other candidate but it choked on blake3's assembly.)

After this, it will probably work perfectly fine, but until we have run the CI we won't be certain of it. It'd be worth adding a way to exercise it before a real release, e.g. a non-publishing job that builds the musl .node and runs the Alpine smoke test on PRs.

@JereSalo

Copy link
Copy Markdown

Created PR pointing to this one with the mentioned fix.

…pine (#237)

* ci(node): build musl addon with rust-musl-cross so it links against musl

The plain cross-build (musl-tools + CC=musl-gcc) silently produced a
glibc-linked .so (NEEDED libc.so.6 / ld-linux-x86-64.so.2) that fails to
load on Alpine, because musl-gcc only wraps the host glibc gcc and does not
set rustc's linker. Build inside messense/rust-musl-cross, which ships a
real musl-targeting GNU toolchain. Verified on ubuntu-24.04 that the
resulting .node loads on node:20-alpine.

The image is pinned by digest rather than the mutable :x86_64-musl tag
because this step runs in the provenance-signed publish pipeline; the
third-party toolchain that builds the shipped binary must be immutable.

* fix unbalanced backtick in build_library doc comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants