Repository navigation
fix(dev-server): start a large Cloudflare app's dev server on remote runners - #265
Merged
Merged
Conversation
mikn
force-pushed
the
mikn/dev-server-store-links
branch
from
October 7, 2026 21:21
d1b8405 to
03971fe
Compare
…runners Four defects kept an application with thousands of declared files and the Cloudflare Vite plugin from starting under ts_dev_server on remote runners: - npm view entries were root symlinks to store trees, which remote execution copies in place, so a package (e.g. @cloudflare/vite-plugin) lost the dependency links beside its store directory (e.g. miniflare). Every entry is now a declared symlink to its store. - oj's fs.watch matched every event against every watcher, opening both files, on the notify thread that also registers new watches. With vite-plugin-bazel watching each entry under bazel-bin (46k watches), that thread never caught up and the plugin host blocked in fs.watch before configureServer returned. oj/oj-deno-fs-events-indexed.patch indexes dispatch by path, ancestor and file identity (upstream: lovablelabs/oj#320). - vite-plugin-bazel realpathed every declared file on each watcher event that missed its index (9M realpath calls). A declared file now only matches a canonical query. - oj 0.2.5's Start fallback renderer handed a dependency's extensionless relative import to Node and answered 500 with ERR_MODULE_NOT_FOUND. oj moves to 0.2.16, which fixed it in 0.2.9. oj 0.2.16 swaps deno_snapshots and deno_process for its oj_deno_snapshots and oj_deno_process forks; V8, deno_core and Cranelift stay pinned. The snapshot source patch moves to oj_deno_snapshots and also reads the fork's new CARGO_MANIFEST_DIR lookup at run time, which the process wrapper otherwise rejects as an embedded absolute path. The code cache and plugin-resolved file HMR patches are re-cut against 0.2.16's sources, the fs.watch patch applies unchanged, and the Start backport is dropped. The npm analysis tests check each view entry as an alias with its store in the runfiles: Starlark exposes no target text for an unresolved symlink. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mikn
force-pushed
the
mikn/dev-server-store-links
branch
from
October 7, 2026 22:13
03971fe to
1d3eb53
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On remote execution,
ts_dev_servercan't start a large application that uses@cloudflare/vite-plugin. Four defects stack:.pnpm/<store>/node_modules/.@cloudflare/vite-pluginthen fails withERR_MODULE_NOT_FOUNDforminiflare.fs.watchstalls the plugin host.oj_deno_runtime(ops/fs_events.rs) matches every filesystem event against every live watcher withsame_file::is_same_file, which opens both files. It does this on notify's inotify thread, which drains every pending event before it registers the next watch. vite-plugin-bazel'sBazelWatcheropens one watch per directory and per linked file under bazel-bin, 46,099 watches on a remote runner. The notify thread spins at 100% CPU, the plugin host's next synchronousfs.watchnever returns, andconfigureServernever finishes. The server never opens its port.realpathSynccalls, and the JS thread stays at 100% CPU.index.ts→./button) to plain Node, which answers 500 withERR_MODULE_NOT_FOUND. Upstream fixed this in oj 0.2.9.Defects 2 and 3 each hang startup alone: fixing only one still never serves a page.
Observed in production
Not observed in production. The ruleset builds development tooling and has no production runtime. All four were found starting a downstream application's dev server, plus a real-route test, on a remote runner.
Gain
On a remote runner, that application's dev server now starts and renders a real route, where before it hung until the test was killed. Its smoke and real-route tests together pass in 214.3 s on oj 0.2.16. With only the oj fix or only the resolver fix applied, it still hangs. Registering 20,000 watches on a patched oj build takes about 1.4 s; an unpatched build never initializes the host within 55 s. Local runs gain the same fixes; nothing measured them separately.
What changes
ts/private/ts_dev_server.bzlctx.actions.declare_symlink) to its store's runfiles path. A store tree that is not itself a symlink joins the runfiles explicitly.oj/oj-deno-fs-events-indexed.patch,oj_deno_runtimeannotation(dev, inode), so each event reaches only its own watchers. Upstream: lovablelabs/oj#320.ops/fs_events.rsis unchanged through oj 0.2.16, so a version bump would not fix it.vite/src/resolver.tsoj/Cargo.toml,oj/Cargo.lock,MODULE.bazel, docs)oj_deno_snapshotsandoj_deno_processforks in place ofdeno_snapshots/deno_process; V8 150.4.0, deno_core 0.412.0 and Cranelift 0.117.2 stay pinned. The lockfile moves only the oj crates.oj-js-code-cache-boundedandoj-server-file-hmrare re-cut against 0.2.16 (oj_server splitlib.rsinto modules; 0.2.16 strips?v=/?t=from cache keys but still keys Vite's timestamped config paths and never trims). The snapshot-source patch moves tooj_deno_snapshotsand also reads its newCARGO_MANIFEST_DIRlookup at run time, because the process wrapper rejects the embedded absolute path.oj-deno-fs-events-indexedapplies unchanged. The rest apply unchanged to the same crate versions.tests/npm/direct_version_tests.bzlprevious_dev_npm_testalready did. Starlark exposes no target text for an unresolved symlink:content,argvandsubstitutionsareNoneon its action under Bazel 9.2.0.vite/tests/resolution_parity_test.mjschangelog.d/Verification
On a remote runner:
bazel test --config=ci //vite/... //tests/dev_server/... //tests/dev_server_assets/... //tests/npm/... //tests/hmrsocket/... //oj/...: 138 of 138 pass in 360 s.bazel test --config=ci //oj/...with oj built from scratch: 4 of 4 pass in 256 s.Risks
dev_inherited_typesanalysis tests no longer prove which store a view entry links to, only that the expected store is in its runfiles. The link target is stillrlocation_pathof the store the old code linked to.BazelWatcherstill opens a watch for every linked file; the fixes make that fast enough, not smaller.🤖 Generated with Claude Code