Skip to content

Desktop: broker owns the /events SSE live mirror (Phase 3 complete) - #74

Merged
zkann merged 1 commit into
mainfrom
desktop-broker-sse
Jun 18, 2026
Merged

Desktop: broker owns the /events SSE live mirror (Phase 3 complete)#74
zkann merged 1 commit into
mainfrom
desktop-broker-sse

Conversation

@zkann

@zkann zkann commented Jun 18, 2026

Copy link
Copy Markdown
Owner

The broker now serves the /events live-mirror stream itself instead of forwarding it — the last read/mirror piece, so Phase 3 is complete: the broker owns the entire live mirror.

What it does

Mirrors dashboard_app.event_stream:

  • a snapshot on connectplate, inflight, pending, queue, runs, each a bare array (the SSE frame shape, distinct from the GET endpoints' {key: ...} wrapper), built from the Node store + liveness;
  • a fresh snapshot whenever SQLite's data_version moves or a file/liveness signal changes (run liveness, session liveness, pending/ + queue/ mtimes — the changes a DB write wouldn't move);
  • a 10s heartbeat.

Holds one read-only connection so data_version is comparable across polls; tears down the connection + timer on client disconnect.

Verification

Against the live FastAPI: the initial snapshot frames (all five events) match, and adding a task re-emits an updated plate frame (change detection confirmed). 33 node --test.

Remaining

  • settings (env-detected terminal) stays forwarded — permanent.
  • The action/write endpoints (run / prepare / put-back / done / dismiss / autonomy / open-session) are Phase 4 — the broker doesn't own writes yet.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added a token-protected Server-Sent Events stream that sends real-time status frames (plate, inflight, pending, queue, runs) and periodic heartbeats, updating when underlying state changes.
  • Bug Fixes
    • Improved access control responses for the stream (returns clear 401 on missing/invalid tokens) and safer handling of streaming failures.
  • Tests
    • Added coverage for snapshot framing, correct event data formatting, behavior when state storage is missing, and signal changes as markdown items are added.
  • Chores
    • Extended the desktop syntax-check script to validate sse.js.

@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ec1b6e6a-9436-404f-8daf-eab76f581973

📥 Commits

Reviewing files that changed from the base of the PR and between 78805b0 and 65e4748.

📒 Files selected for processing (4)
  • desktop/broker.js
  • desktop/package.json
  • desktop/sse.js
  • desktop/sse.test.js
🚧 Files skipped from review as they are similar to previous changes (3)
  • desktop/package.json
  • desktop/sse.test.js
  • desktop/broker.js

📝 Walkthrough

Walkthrough

Adds desktop/sse.js, a new module that streams five live-mirror SSE frames (plate, inflight, pending, queue, runs) via polling against SQLite data_version and directory mtimes. The broker gains a token-gated GET /events handler delegating to this module. Tests cover snapshot frame structure and signal change detection.

Changes

SSE live-mirror endpoint

Layer / File(s) Summary
SSE module: snapshot, signals, and stream
desktop/sse.js
Defines configuration intervals, the SSE formatter, snapshot() generator for five live-mirror frames, dirSig() for .md file mtime signatures, signals() for composite change detection from liveness and directory state, and createEventStream() implementing SSE headers, lazy SQLite connection with retry on transient failures, polling for data_version and signal changes, heartbeat emissions, and cleanup on close/error.
Broker /events route wiring
desktop/broker.js, desktop/package.json
Imports sse, adds a GET /events branch token-gated via the t query param (returning 401 JSON on failure), delegates to sse.createEventStream on success with fallback 500 error handling, and adds node --check sse.js to the syntax-check script.
SSE module tests
desktop/sse.test.js
Test harness setup, seeded() helper builds a temp SQLite-backed state; tests assert five properly terminated frames with bare-array payloads and expected plate content; a second test verifies snapshot resilience when state.db is missing; a third test verifies that signals() output changes after writing files into pending/ and queue/.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • zkann/smbos#69: Both PRs modify desktop/broker.js to handle SSE on /events — the retrieved PR adds an unbuffered proxy pass-through, while this PR replaces that with a locally broker-handled, token-gated SSE stream.
  • zkann/smbos#70: Both PRs apply the same constant-time tokenOk / t-param token-gating pattern in desktop/broker.js for broker-handled routes — the retrieved PR for /api/plate and /api/queue, this PR for the new GET /events route.
  • zkann/smbos#73: This PR's new desktop/sse.js signals and stream logic directly consumes liveness exports like activeRuns and inflightWithLiveness added in the retrieved PR, and desktop/broker.js routes SSE alongside the broker's liveness-based endpoints.

Poem

🐇 Hop hop, a new stream is born today,
Five frames of data sent SSE way,
plate, inflight, queue in a row,
Heartbeats tick as the signals flow,
The broker checks your token with care,
Then live-mirror snapshots fill the air! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 71.43% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: implementing direct broker ownership of the /events SSE live-mirror stream as part of Phase 3 completion. It is specific, concise, and clearly communicates the primary objective.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch desktop-broker-sse

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a65cc87b97

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread desktop/sse.js Outdated
})
// One held read-only connection so data_version is comparable across polls (it's per-connection).
let db = null
try { db = new DatabaseSync(path.join(sopDir, 'state.db'), { readOnly: true }); db.exec('PRAGMA busy_timeout = 2000') } catch (_) { db = null }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reopen the SSE reader when the database appears

When /events is opened on a fresh SOP dir before state.db exists, this read-only open falls into the catch and leaves db null for the lifetime of the stream. Later forwarded writes/imports can create and populate state.db, but dataVersion() will keep returning 0 and the DB contents are not part of signals(), so the live mirror never emits the new plate/run data until the browser reconnects. Retry/open the reader when the DB appears, or create/migrate it the way the FastAPI stream did.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: the held data_version connection now opens LAZILY (ensureDb) and re-opens on error, so a state.db that does not exist at connect (fresh SOP dir) or is recreated mid-stream gets picked up -- data_version then observes the new commits and re-emits, instead of pinning to 0 / a stale handle. (snapshot()/signals() already re-read fresh each poll, so they pick up the appeared db; this closes the data_version side.) Added a snapshot-resilient-to-missing-db test.

Comment thread desktop/sse.js Outdated
Comment on lines +69 to +70
let lastSig = signals(sopDir)
for (const frame of snapshot(sopDir)) res.write(frame) // initial snapshot

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard the initial SSE snapshot from store failures

If state.db exists but the schema is not ready or readable yet (for example during first creation/migration, or an older/partial DB), signals() or snapshot() can throw no such table before any timer-level catch runs. Because this happens inside the HTTP handler without the served-read try/catch, one /events request can terminate the broker process instead of failing or retrying the stream.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already addressed in the hardening push that landed alongside this review: the initial snapshot/signals read is now inside a try/catch in createEventStream (with the close/error cleanup attached BEFORE the first read, and the broker call defensively wrapped), so a "no such table" during first-creation/migration tears the stream down cleanly instead of crashing the broker.

@zkann

zkann commented Jun 18, 2026

Copy link
Copy Markdown
Owner Author

Adversarial self-review hardening (folded into the latest push):

  • [P2] db leak + uncaught throw on a failed initial snapshot: createEventStream opened the held connection and read the first snapshot BEFORE attaching the close/error cleanup, and the broker call was unguarded -- a throw (corrupt/mid-recreate state.db) leaked the connection and could crash the broker. Now: cleanup handlers attach BEFORE any read, the snapshot/interval setup is wrapped in try/catch (tears down + ends the response), and the broker call is defensively wrapped too.
  • [P3] dirSig nanosecond mtime: switched to statSync({bigint:true}).mtimeNs (was mtimeMs) to match Python st_mtime_ns -- catches two sub-millisecond rewrites of a pending/queue file.
  • [P3] collision-proof signals: signals() now JSON-encodes structured [id,state]/[name,mtime] values instead of joining on :/,/#/| -- an id or filename containing a delimiter can no longer alias to "no change".
  • Confirmed clean by the review: res.on(close) reliably fires + cleans up; res.write after close does not throw; held readOnly data_version observes writes repeatedly; no missing SSE frame (SPA listens for exactly the 5 + heartbeat the broker emits; settings/procedures are GET, not streamed).
    33 node --test; snapshot parity + change detection re-smoked.

@zkann
zkann force-pushed the desktop-broker-sse branch 2 times, most recently from aa42c97 to 78805b0 Compare June 18, 2026 15:07

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@desktop/sse.js`:
- Line 70: The snapshot function calls at both locations (the for loop iterating
over snapshot(sopDir) and the other snapshot call around lines 77-80) lack error
handling, which means any thrown exceptions during store or liveness reads will
propagate up and crash the broker process. Wrap both snapshot(sopDir) calls in
try-catch blocks to gracefully handle any errors that occur during snapshot
generation, ensuring exceptions are logged but do not terminate the process or
handler execution.
- Around line 55-66: The catch block on line 62 silently swallows all exceptions
when creating the DatabaseSync instance by setting db to null, which causes
dataVersion() to always return 0 and disables change-triggered refreshes.
Instead of using catch (_) { db = null }, fail fast by re-throwing the error or
responding with an appropriate error status code to the client. This ensures
that database connection failures are properly surfaced rather than silently
degrading the SSE stream's functionality.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ca7b7385-8ae2-4ed6-b655-01da216a8c62

📥 Commits

Reviewing files that changed from the base of the PR and between 63a972f and a65cc87.

📒 Files selected for processing (4)
  • desktop/broker.js
  • desktop/package.json
  • desktop/sse.js
  • desktop/sse.test.js

Comment thread desktop/sse.js
Comment thread desktop/sse.js Outdated
The broker now serves the /events live-mirror stream itself, instead of forwarding it --
the last read/mirror piece. Mirrors dashboard_app.event_stream: a snapshot on connect
(plate, inflight, pending, queue, runs -- each a BARE array, the SSE shape, using the
Node store + liveness), a fresh snapshot whenever SQLite's data_version moves OR a file/
liveness signal changes, and a 10s heartbeat. Holds one read-only connection so
data_version is comparable across polls; tears down on client disconnect.

Verified against the live FastAPI: the initial snapshot frames (all five events) match
byte-for-byte, and adding a task re-emits an updated plate frame (change detection works).

- desktop/sse.js (+ tests): snapshot(), signals() (run/session liveness + pending/queue
  mtimes -- the changes a DB write wouldn't move), createEventStream().
- desktop/broker.js: GET /events is token-gated then streamed (before the JSON SERVED map).

With this the broker owns the WHOLE live mirror (every read + the stream). Still forwarded:
settings (env-detected terminal) and the action/write endpoints (Phase 4, next).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@zkann
zkann force-pushed the desktop-broker-sse branch from 78805b0 to 65e4748 Compare June 18, 2026 15:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
desktop/sse.js (1)

101-104: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Guard periodic snapshot emission in the timer callback.

Line 103 still calls snapshot(sopDir) without a local guard inside setInterval. If store/liveness read throws there, the outer try at Lines 90-110 will not catch it (async callback), which can surface as an uncaught exception.

Suggested fix
       if (dv !== lastDv || sig !== lastSig) {
         lastDv = dv; lastSig = sig
-        for (const frame of snapshot(sopDir)) res.write(frame)  // all frames on any change
+        try {
+          for (const frame of snapshot(sopDir)) res.write(frame)  // all frames on any change
+        } catch (_) {
+          cleanup()
+          try { res.end() } catch (_) { /* already closed */ }
+          return
+        }
       }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@desktop/sse.js` around lines 101 - 104, The snapshot(sopDir) call on line 103
inside the setInterval callback is not guarded by a try-catch block, which means
if it throws an error, it will not be caught by the outer try block at Lines
90-110 and will become an uncaught exception. Wrap the snapshot(sopDir) call and
the subsequent res.write(frame) loop in a try-catch block within the setInterval
callback to handle any errors that may occur during periodic snapshot emission,
ensuring errors are properly caught and handled without crashing the process.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@desktop/sse.js`:
- Around line 101-104: The snapshot(sopDir) call on line 103 inside the
setInterval callback is not guarded by a try-catch block, which means if it
throws an error, it will not be caught by the outer try block at Lines 90-110
and will become an uncaught exception. Wrap the snapshot(sopDir) call and the
subsequent res.write(frame) loop in a try-catch block within the setInterval
callback to handle any errors that may occur during periodic snapshot emission,
ensuring errors are properly caught and handled without crashing the process.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7390e8ad-17e8-45ee-964c-7d20d423c73f

📥 Commits

Reviewing files that changed from the base of the PR and between a65cc87 and 78805b0.

📒 Files selected for processing (4)
  • desktop/broker.js
  • desktop/package.json
  • desktop/sse.js
  • desktop/sse.test.js
✅ Files skipped from review due to trivial changes (1)
  • desktop/package.json
🚧 Files skipped from review as they are similar to previous changes (2)
  • desktop/broker.js
  • desktop/sse.test.js

@zkann
zkann merged commit 591904c into main Jun 18, 2026
7 checks passed
@zkann
zkann deleted the desktop-broker-sse branch June 18, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant