Core: Fix manager overload and false "Server timed out" disconnects during large test runs

[yannbf]

Closes #

What I did

When you run tests on a project with many stories, the manager shows "Server timed out — Please restart your Storybook server and reload the page" partway through the run. The server is actually fine — the manager kills its own connection because it can't keep up. Step by step:

1. During a test run, status updates stream to the manager several times per second.
2. Since Sidebar: Add status-based filtering with refactored status architecture #34339, every one of those updates rebuilds the entire index twice: once in the onAllStatusChange handler, and once more via recomputeStatusFilter() → experimental_setFilter(), which runs its own setIndex(). On a ~3,600-story project the manager can't process events as fast as they arrive, so its UI falls minutes behind the actual run.
3. Unprocessed messages pile up in a queue. The heartbeat watchdog closes the socket (code 3008) when no ping is processed for 20 seconds — but the pings are fine, they're just stuck behind the queued messages. Hence the toast, even though the server never went down. Core: Reset WebsocketTransport heartbeat on every message, not just ping #35422 helped but didn't cure it: the watchdog still fires while the backlog drains after the tests have already finished.
4. Amplifying all of this, every store event from the vitest child process reached each browser twice — once broadcast directly by channel.emit, and once more by the store leader's own event forwarding.

Three fixes, one per problem:

• Rebuild the index once, not twice, and at most once per second. The registered status filter reads statuses live from the store whenever it's applied, so re-registering it on every status change achieves nothing except the second rebuild. Removed. The remaining rebuild is throttled (leading + trailing, so the final state always lands).
• Don't mistake a busy main thread for a dead server. If the heartbeat timer fires noticeably later than scheduled, the thread was blocked at the deadline and queued pings may simply not have been processed yet — grant one grace window instead of closing. On-time fires still close with 3008 exactly as before.
• Send each store event once. Child store events now go through a new Channel.receive() — dispatch to local listeners without re-broadcasting — leaving the store leader's forward as the single copy browsers receive.

Measured on a 3,583-test project (coverage on, manager attached):

	before	after
Run outcome	"Server timed out" at ~300 tests	Completes, no disconnect
Testing widget progress	Minutes behind the run	Real time
Data sent to each browser	~3.4 GB per run	~1.2 GB
Renderer heap peak	~690 MB	~80 MB

The canary built from this branch was verified end-to-end against the same project: full run completed, zero disconnects on both the manager and preview sockets.

Commit structure

The first commit adds seven tests, six of which fail on next — they document the bug (bounded rebuilds under a status-update burst, no filter re-registration, no close on a late-firing heartbeat, and store events reaching clients exactly once: the last one uses a real leader store, so it fails both when events are duplicated and if they ever stop being forwarded at all). The second commit is the fix that makes them pass. The suite was hardened with mutation testing — e.g. removing the throttle's trailing edge (which would silently lose the final second of statuses in every run) fails a test deterministically.

Known pre-existing issues in the touched code, left for follow-ups

• Two concurrent setIndex callers can race (a status-driven rebuild reading a not-yet-committed index from a concurrent fetchIndex can win the commit order). This predates the PR — it used to happen per status event, now at most once per second — and self-heals on the next index invalidation. A generation guard in setIndex would fix it for all callers.
• bootTestRunner registers process.on(exit/SIGINT/SIGTERM) and a FATAL_ERROR subscription on every child (re)boot without ever removing them, so crash-looping children accumulate listeners.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

• stories
• unit tests
• integration tests
• end-to-end tests

Manual testing

1. Use a project with a large number of story tests (this reproduces reliably around 3,500; small sandboxes only show the halved event traffic, not the disconnect).
2. Open the manager, enable coverage in the testing widget, run all tests.
3. Before: the widget's progress falls far behind the terminal's vitest output, and a "Server timed out" toast appears mid-run or shortly after vitest finishes. After: progress tracks in real time and the run completes without a toast.
4. To see the dedupe: devtools → Network → the storybook-server-channel WS connection — each UNIVERSAL_STORE:* frame should appear once during a run, not twice back to back.

Documentation

• Add or update documentation reflecting your changes
• If you are deprecating/removing a feature, make sure to update
  MIGRATION.MD

Checklist for Maintainers

• When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts
• Declare whether manual QA will be needed for this PR during the next release, through qa:needed or qa:skip

This pull request has been released as version 0.0.0-pr-35429-sha-df3a9be0. Try it out in a new sandbox by running npx storybook@0.0.0-pr-35429-sha-df3a9be0 sandbox or in an existing project with npx storybook@0.0.0-pr-35429-sha-df3a9be0 upgrade.

More information

Published version	0.0.0-pr-35429-sha-df3a9be0
Triggered by	@yannbf
Repository	storybookjs/storybook
Branch	yann/fix-test-run-status-flood
Commit	df3a9be0
Datetime	Mon Jul 13 14:10:52 UTC 2026 (1783951852)
Workflow run	29256832947

To request a new release of this pull request, mention the @storybookjs/core team.

core team members can create a new canary release here or locally with gh workflow run --repo storybookjs/storybook publish.yml --field pr=35429

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added a new channel event “receive” path to deliver externally-arrived events to local listeners.
  • Throttled status-driven story index rebuilds during rapid status updates.

• Bug Fixes

  • Improved WebSocket heartbeat handling during main-thread stalls, including correct timeout/grace behavior.
  • Prevented universal store test events from being re-broadcast when already forwarded, while ensuring leader forwarding occurs exactly once.

• Tests

  • Added coverage for universal store routing, WebSocket starvation/grace scenarios, and bounded/throttled rebuild behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 428bb1a2-cb4a-4636-ae91-0248dc4cede0

📥 Commits

Reviewing files that changed from the base of the PR and between fb2a3e2 and 050ea58.

📒 Files selected for processing (7)

• code/addons/vitest/src/node/boot-test-runner.test.ts
• code/addons/vitest/src/node/boot-test-runner.ts
• code/core/src/channels/main.ts
• code/core/src/channels/websocket/index.test.ts
• code/core/src/channels/websocket/index.ts
• code/core/src/manager-api/modules/stories.ts
• code/core/src/manager-api/tests/stories.test.ts

🚧 Files skipped from review as they are similar to previous changes (5)

• code/core/src/channels/main.ts
• code/core/src/manager-api/tests/stories.test.ts
• code/core/src/manager-api/modules/stories.ts
• code/core/src/channels/websocket/index.ts
• code/core/src/channels/websocket/index.test.ts

---

📝 Walkthrough

Walkthrough

This PR adds local delivery for universal store events in the Vitest boot path, changes websocket heartbeat timeout handling to tolerate delayed timers, and throttles story index rebuilds triggered by status updates.

Changes

Universal Store Event Routing

Layer / File(s)	Summary
Channel.receive API code/core/src/channels/main.ts	Adds receive(event) to deliver an incoming event to local listeners without sending it back through transports.
Route universal store events through receive code/addons/vitest/src/node/boot-test-runner.ts, code/addons/vitest/src/node/boot-test-runner.test.ts	Defines UNIVERSAL_STORE_EVENT_NAMES and routes matching child-process events through channel.receive; tests verify store events bypass transport.send while other events still flow through the transport path.

WebSocket Heartbeat Starvation Grace

Layer / File(s)	Summary
Heartbeat grace window implementation and tests code/core/src/channels/websocket/index.ts, code/core/src/channels/websocket/index.test.ts	Adds HEARTBEAT_TIMEOUT and HEARTBEAT_STARVATION_SLACK, updates heartbeat timeout handling to re-arm once on late firings before closing with code 3008, and adds tests for late firing, on-time timeout, grace reset, and grace exhaustion.

Status-Change Rebuild Throttling

Layer / File(s)	Summary
Throttled onAllStatusChange rebuild code/core/src/manager-api/modules/stories.ts, code/core/src/manager-api/tests/stories.test.ts	Adds STATUS_CHANGE_REBUILD_THROTTLE and replaces per-update status recomputation with a throttled rebuild that reapplies the current index and refs; tests cover filter reapplication and bounded rebuild frequency during bursts.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ChildProcess
  participant BootTestRunner
  participant Channel
  participant Transport
  ChildProcess->>BootTestRunner: message (UNIVERSAL_STORE event)
  BootTestRunner->>BootTestRunner: check UNIVERSAL_STORE_EVENT_NAMES
  BootTestRunner->>Channel: channel.receive(event)
  Channel->>Channel: handleEvent (local listeners only)
  ChildProcess->>BootTestRunner: message (other-event)
  BootTestRunner->>Channel: channel.emit(type, args)
  Channel->>Transport: transport.send(event)

Loading

Possibly related PRs

• storybookjs/storybook#35422: Updates the same websocket heartbeat logic and test area in code/core/src/channels/websocket/index.ts.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

code/core/src/manager-api/tests/stories.test.ts (1)

2064-2111: 🩺 Stability & Availability | 🔵 Trivial | 🏗️ Heavy lift

Real-timer-based throttle assertion risks CI flakiness.

The stress test waits on real wall-clock time (setTimeout(resolve, 1100)) to let the 1000ms throttle window settle before asserting setIndexSpy.mock.calls.length <= 4. Under a loaded CI runner, GC pauses or scheduling delays could push the burst's synchronous processing time (50 fullStatusStore.set calls, each triggering real listener/index logic) past the throttle window, causing intermittent extra rebuild cycles and flaky failures — the reverse can also happen if the sleep resolves before the trailing edge fires. Using vi.useFakeTimers() with vi.advanceTimersByTimeAsync() would make the throttle boundary deterministic instead of relying on generous real-time slack.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/manager-api/tests/stories.test.ts` around lines 2064 - 2111,
The stress test in stories.test.ts is using a real 1100ms sleep to wait for the
throttle window, which can be flaky under CI timing jitter. Update the test
around initStories, api.setIndex, and the setIndexSpy/assertion block to use
vi.useFakeTimers() and advance time deterministically with
vi.advanceTimersByTimeAsync() instead of wall-clock waiting. Make sure the burst
of fullStatusStore.set updates is followed by explicit timer advancement so the
trailing-edge rebuild is observed before asserting the bounded call count.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@code/core/src/manager-api/tests/stories.test.ts`:
- Around line 2064-2111: The stress test in stories.test.ts is using a real
1100ms sleep to wait for the throttle window, which can be flaky under CI timing
jitter. Update the test around initStories, api.setIndex, and the
setIndexSpy/assertion block to use vi.useFakeTimers() and advance time
deterministically with vi.advanceTimersByTimeAsync() instead of wall-clock
waiting. Make sure the burst of fullStatusStore.set updates is followed by
explicit timer advancement so the trailing-edge rebuild is observed before
asserting the bounded call count.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cfe515fe-5d4f-43bd-afc9-48391b633f5f

📥 Commits

Reviewing files that changed from the base of the PR and between d7a8cbc and fb2a3e2.

📒 Files selected for processing (7)

• code/addons/vitest/src/node/boot-test-runner.test.ts
• code/addons/vitest/src/node/boot-test-runner.ts
• code/core/src/channels/main.ts
• code/core/src/channels/websocket/index.test.ts
• code/core/src/channels/websocket/index.ts
• code/core/src/manager-api/modules/stories.ts
• code/core/src/manager-api/tests/stories.test.ts

🚧 Files skipped from review as they are similar to previous changes (4)

• code/core/src/channels/main.ts
• code/core/src/manager-api/modules/stories.ts
• code/addons/vitest/src/node/boot-test-runner.test.ts
• code/core/src/channels/websocket/index.ts