Skip to content

Desktop: Phase 3 (step 1) broker serves the static reads (plate, queue) - #70

Merged
zkann merged 1 commit into
mainfrom
desktop-broker-reads
Jun 17, 2026
Merged

Desktop: Phase 3 (step 1) broker serves the static reads (plate, queue)#70
zkann merged 1 commit into
mainfrom
desktop-broker-reads

Conversation

@zkann

@zkann zkann commented Jun 17, 2026

Copy link
Copy Markdown
Owner

Start of Phase 3 of the switchover: the broker stops forwarding the static reads and answers them itself from the SQLite work-state + plain files.

What moved

  • /api/plate (SQLite task table) and /api/queue (queue .md files) are now served by the broker via desktop/store.js (node:sqlite, built into Node — no native module).
  • The broker owns the token gate for what it serves (FastAPI's check never runs on a broker-served response) and keeps the Host guard. Everything else forwards unchanged.

Scope / honesty

  • Parity-verified: a live check against the running FastAPI confirms the broker's /api/plate and /api/queue output matches byte for byte.
  • Deferred deliberately: settings / procedures / pending follow in later increments. The parity check caught that /api/settings reads an environment-detected terminal (iterm) and serializes a float budget, so it isn't a pure static read — it stays forwarded rather than ship a subtly-divergent port.
  • The liveness-bearing reads (inflight, runs) and the SSE live mirror keep forwarding until the Phase 5 native layer migrates the flock/pid liveness (a real architectural coupling, not an oversight).

Tests

node --test: 16 (store unit tests + a broker served-vs-forwarded test). Plus the live FastAPI parity check above.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added locally served dashboard endpoints for plate and queue, protected by a token. Invalid or missing tokens return 401; valid tokens return results from the local dashboard data.
  • Tests

    • Added integration tests for token-gated local serving (including “fail-closed” behavior) and verified unserved routes are still proxied upstream.
    • Added unit tests for SQLite-backed plate reads and filesystem-backed queue parsing, including frontmatter comment handling.
  • Chores

    • Expanded desktop syntax checks to include the new store module and upgraded Electron.

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 206a5b9d-eb53-4e24-beb8-ff9c44a2d304

📥 Commits

Reviewing files that changed from the base of the PR and between 6e7183b and 6e19d7c.

⛔ Files ignored due to path filters (1)
  • desktop/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (6)
  • desktop/broker.js
  • desktop/broker.test.js
  • desktop/main.js
  • desktop/package.json
  • desktop/store.js
  • desktop/store.test.js
🚧 Files skipped from review as they are similar to previous changes (4)
  • desktop/main.js
  • desktop/package.json
  • desktop/store.test.js
  • desktop/broker.js

📝 Walkthrough

Walkthrough

Adds a new store.js module that reads plate tasks from SQLite and queued items from markdown frontmatter. Extends createBroker with a sopDir parameter and a token-gated local serving path for /api/plate and /api/queue, while continuing to forward all other routes to FastAPI. Wires sopDir through main.js and expands the check script.

Changes

Phase 3 broker: local store reads for /api/plate and /api/queue

Layer / File(s) Summary
store.js plate and queue implementations
desktop/store.js, desktop/store.test.js
New plate(sopDir) queries SQLite state.db for status='waiting' tasks ordered by priority (desc), creation time (asc), and ID; queue(sopDir) scans queue/*.md files parsing YAML-like frontmatter (skipping # comment lines) filtered to status: queued. A parseFrontmatter helper and withDb read-only DB utility support both. Tests cover ordering, status filtering, missing-DB/missing-dir edge cases, and project basename extraction.
Broker token-gated local serving and integration tests
desktop/broker.js, desktop/broker.test.js
createBroker gains sopDir parameter and a SERVED map for /api/plate and /api/queue; incoming GET requests are gated by constant-time tokenOk check on the t query param, delegating to store on success or returning JSON 401 for bad/missing tokens or 500 on store read failures. Non-served routes continue to forward to FastAPI. Integration tests assert 401 without token, 200 with seeded data and matching token, no upstream contact for served routes, and continued forwarding for unserved routes. Fail-closed tests confirm 401 when .dashboard-token is missing or empty.
main.js and package.json wiring
desktop/main.js, desktop/package.json
main.js destructures sopDir from ./resolve and passes sopDir() into createBroker at broker startup. package.json check script extended to validate store.js syntax alongside existing modules; electron devDependency bumped to ^42.4.1.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Broker
  participant store_js as store.js
  participant FastAPI

  rect rgba(100, 149, 237, 0.5)
    Note over Client,store_js: Token-gated local serve (/api/plate, /api/queue)
    Client->>Broker: GET /api/plate?t=TOKEN
    Broker->>Broker: tokenOk check via timingSafeEqual
    alt bad/missing token
      Broker-->>Client: 401 {"error":"Unauthorized"}
    else store read error
      Broker->>store_js: plate(sopDir)
      store_js-->>Broker: throws
      Broker-->>Client: 500 {"error":"..."}
    else success
      Broker->>store_js: plate(sopDir)
      store_js->>store_js: open state.db read-only, query waiting tasks
      store_js-->>Broker: rows[]
      Broker-->>Client: 200 payload
    end
  end

  rect rgba(144, 238, 144, 0.5)
    Note over Client,FastAPI: Forwarded routes (unchanged)
    Client->>Broker: GET /api/runs
    Broker->>FastAPI: proxy request (filtered headers + Host guard)
    FastAPI-->>Client: upstream response
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • zkann/smbos#69: Introduced the Phase 2 broker facade in the same desktop/broker.js and desktop/main.js files; this PR extends that broker directly with Phase 3 local serving.
  • zkann/smbos#32: Both PRs implement token-gated dashboard reads for /api/plate using a t query token and SOP directory token source; the main PR serves locally from the broker, while PR #32 serves from FastAPI upstream.

Poem

🐇 Hippity-hop through the broker's new door,
A token to knock, a store on the floor,
SQLite plates and queued markdown files,
No proxy needed — served in local styles!
Phase 3 is here, with a timingSafeEqual,
This rabbit's dashboard reads feel quite ideal. 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes the main change: transitioning the broker to serve static API endpoints (/api/plate and /api/queue) instead of forwarding them, which is the core objective of Phase 3 step 1.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch desktop-broker-reads

Comment @coderabbitai help to get the list of available commands and usage tips.

@zkann

zkann commented Jun 17, 2026

Copy link
Copy Markdown
Owner Author

Adversarial self-review hardening (folded into the latest push). No security holes (token gate equivalent to FastAPI's, missing-token fails closed, no path-bypass of the gate). Two P2 fixes:

  • SQLite busy_timeout: store reads now set PRAGMA busy_timeout=2000, matching state_store.connect, so a read concurrent with a Python writer / WAL checkpoint / ALTER migration waits out the lock instead of returning a 500 where FastAPI would have waited.
  • Frontmatter comment parity: the JS frontmatter parser now skips #-comment lines like smbos_lib.parse_frontmatter, removing a latent queue-parity divergence.
    Added tests: a #-comment queue fixture, and a fails-closed test (missing/empty token file -> 401 on a served read, not allowed). 18 node --test cases pass; live FastAPI parity still exact.

@zkann
zkann force-pushed the desktop-broker-reads branch from 0e315f4 to 6e7183b Compare June 17, 2026 22:59

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0e315f48f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread desktop/store.js
//
// Each reader mirrors its Python counterpart exactly; the parity test (store == FastAPI) is the gate.

const { DatabaseSync } = require('node:sqlite')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Replace node:sqlite before Electron startup

When the desktop app is launched with the pinned Electron 32.3.3 runtime, this top-level import runs while loading main.js -> broker.js -> store.js; Electron 32 embeds Node 20.18.1, but node:sqlite is only available in newer Node releases, so the require fails before the broker, tray, or window can start. The local node --test path can pass under a newer system Node, but npm start uses Electron's older embedded Node, so the shipped desktop shell is broken until this uses a dependency/runtime Electron actually provides or Electron is upgraded accordingly.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct and shipping-critical, fixed. Electron 32 embeds Node 20.18, but node:sqlite needs Node >= 22.5, so require("node:sqlite") would have crashed the shell at startup (my node --test passed only because it ran under system Node 24). Fix: bumped the Electron pin to ^42.4.1 (embeds Node 24.16). Verified in the actual Electron runtime headlessly: ELECTRON_RUN_AS_NODE=1 electron -e "require(\"node:sqlite\")" -> OK, and store.js loads with its plate/queue exports. No native module / ABI rebuild needed (the reason for keeping node:sqlite over better-sqlite3). Documented the Node>=22.5 / Electron>=42 requirement in store.js.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
desktop/broker.test.js (1)

136-160: ⚡ Quick win

Extend broker integration coverage to include the /api/queue served path.

This test validates /api/plate local serving, but /api/queue is also broker-served and currently lacks broker-level served-vs-forwarded coverage.

Suggested addition
 test('serves /api/plate from the store (token-gated), forwards unknown paths', async () => {
   const d = fs.mkdtempSync(path.join(os.tmpdir(), 'smbos-broker-'))
   fs.writeFileSync(path.join(d, '.dashboard-token'), 'tok')
+  fs.mkdirSync(path.join(d, 'queue'))
+  fs.writeFileSync(
+    path.join(d, 'queue', 'q.md'),
+    '---\nstatus: queued\nsop: nightly\nproject: /tmp/acme\n---\nbody',
+  )

   const db = new DatabaseSync(path.join(d, 'state.db'))
   db.exec(`CREATE TABLE task (id INTEGER PRIMARY KEY, domain TEXT, kind TEXT, subject TEXT, status TEXT,
     priority INTEGER DEFAULT 0, source_ref TEXT, created_at TEXT, updated_at TEXT)`)
   db.prepare("INSERT INTO task(id,domain,kind,subject,status,created_at,updated_at) VALUES(1,'ops','x','on plate','waiting','t','t')").run()
   db.close()

   let forwarded = false
   const upstream = http.createServer((req, res) => { forwarded = true; res.end('up') })
   const upPort = await listen(upstream)
   const broker = createBroker({ targetPort: upPort, sopDir: d })
   const brPort = await listen(broker)

   // served + no token -> 401, never forwarded
   assert.equal((await request(brPort, '/api/plate')).status, 401)

   // served + valid token -> answered from the store
   const ok = await request(brPort, '/api/plate?t=tok')
   assert.equal(ok.status, 200)
   assert.deepEqual(JSON.parse(ok.body).plate.map((r) => r.subject), ['on plate'])

+  const q = await request(brPort, '/api/queue?t=tok')
+  assert.equal(q.status, 200)
+  assert.deepEqual(JSON.parse(q.body).queue, [{ file: 'q.md', sop: 'nightly', project: 'acme' }])

   assert.equal(forwarded, false, 'a served read never hits the upstream')
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@desktop/broker.test.js` around lines 136 - 160, The test function "serves
/api/plate from the store (token-gated), forwards unknown paths" currently only
validates token-gated serving and forwarding behavior for the /api/plate
endpoint, but the broker also serves /api/queue locally and needs the same
coverage. Add additional assertions after the existing /api/plate test cases
(and before the upstream.close() and broker.close() calls) that mirror the same
test pattern for /api/queue: verify it returns 401 without a token, returns 200
with a valid token, and confirms that forwarded remains false when served from
the local store.
desktop/store.test.js (1)

21-30: ⚡ Quick win

Add an explicit fixture for the id ASC tie-break in plate ordering.

The current case validates priority/time ordering, but it does not exercise equal-priority + equal-created_at rows, so the id ASC contract in the test name isn’t actually asserted.

Proposed test delta
   seedTasks(d, [
     { id: 1, subject: 'a', status: 'waiting', priority: 0, created_at: '2026-01-02' },
     { id: 2, subject: 'b', status: 'in_flight', priority: 9 },                 // not waiting -> excluded
     { id: 3, subject: 'c', status: 'waiting', priority: 5, created_at: '2026-01-03' },
     { id: 4, subject: 'd', status: 'waiting', priority: 0, created_at: '2026-01-01' },
+    { id: 5, subject: 'e', status: 'waiting', priority: 0, created_at: '2026-01-01' },
   ])
-  assert.deepEqual(store.plate(d).map((r) => r.subject), ['c', 'd', 'a'])  // c(prio5), then prio0 oldest-first d,a
+  assert.deepEqual(store.plate(d).map((r) => r.subject), ['c', 'd', 'e', 'a'])
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@desktop/store.test.js` around lines 21 - 30, The test name for the plate
function indicates it should order by priority desc, then created_at asc, then
id asc, but the current seedTasks call does not include any rows with equal
priority and equal created_at values. Add additional task fixtures to the
seedTasks array that have matching priority and created_at but different ids,
then update the assert.deepEqual assertion to verify that these equal-priority,
equal-created_at rows are correctly ordered by id in ascending order.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@desktop/store.js`:
- Line 8: The import statement on line 8 uses the `node:sqlite` built-in module,
which is not available in Node.js 20.16.0 that ships with Electron 32.0.0
(requires Node.js 22.5.0+). Replace the `require('node:sqlite')` import with a
compatible SQLite library such as `better-sqlite3` or `sqlite3` that works with
Node.js 20, adjusting the usage of DatabaseSync to match the API of the chosen
alternative library, or alternatively upgrade to Electron 33 or later which
bundles Node.js 22+.

---

Nitpick comments:
In `@desktop/broker.test.js`:
- Around line 136-160: The test function "serves /api/plate from the store
(token-gated), forwards unknown paths" currently only validates token-gated
serving and forwarding behavior for the /api/plate endpoint, but the broker also
serves /api/queue locally and needs the same coverage. Add additional assertions
after the existing /api/plate test cases (and before the upstream.close() and
broker.close() calls) that mirror the same test pattern for /api/queue: verify
it returns 401 without a token, returns 200 with a valid token, and confirms
that forwarded remains false when served from the local store.

In `@desktop/store.test.js`:
- Around line 21-30: The test name for the plate function indicates it should
order by priority desc, then created_at asc, then id asc, but the current
seedTasks call does not include any rows with equal priority and equal
created_at values. Add additional task fixtures to the seedTasks array that have
matching priority and created_at but different ids, then update the
assert.deepEqual assertion to verify that these equal-priority, equal-created_at
rows are correctly ordered by id in ascending order.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 70e1cd0d-f091-4b21-92e2-c7c9442ffa64

📥 Commits

Reviewing files that changed from the base of the PR and between 48d39a8 and 0e315f4.

📒 Files selected for processing (6)
  • desktop/broker.js
  • desktop/broker.test.js
  • desktop/main.js
  • desktop/package.json
  • desktop/store.js
  • desktop/store.test.js

Comment thread desktop/store.js
@zkann
zkann force-pushed the desktop-broker-reads branch from 6e7183b to 1eaa975 Compare June 17, 2026 23:04

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
desktop/broker.test.js (1)

136-160: ⚡ Quick win

Add broker integration coverage for /api/queue local serving.

This integration test validates /api/plate only; Phase 3 serving also includes /api/queue, so wiring/token-gate behavior for that route is currently unverified here.

Possible extension (same test or sibling test)
   // served + valid token -> answered from the store
   const ok = await request(brPort, '/api/plate?t=tok')
   assert.equal(ok.status, 200)
   assert.deepEqual(JSON.parse(ok.body).plate.map((r) => r.subject), ['on plate'])
   assert.equal(forwarded, false, 'a served read never hits the upstream')
+
+  // served queue route should also be local + token-gated
+  fs.mkdirSync(path.join(d, 'queue'))
+  fs.writeFileSync(path.join(d, 'queue', 'q.md'), '---\nstatus: queued\nsop: weekly\n---\nbody')
+  const q = await request(brPort, '/api/queue?t=tok')
+  assert.equal(q.status, 200)
+  assert.deepEqual(JSON.parse(q.body).queue, [{ file: 'q.md', sop: 'weekly', project: '' }])
+  assert.equal(forwarded, false, 'served queue read never hits the upstream')
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@desktop/broker.test.js` around lines 136 - 160, The test only validates
`/api/plate` local serving and token-gating behavior, but Phase 3 also serves
`/api/queue` which lacks verification here. Add test assertions to verify
`/api/queue` follows the same token-gate and local-serving pattern: first insert
a queue record into the task table in the test setup, then add requests to
`/api/queue` without a token (expecting 401), with a valid token (expecting 200
with the queue data from store), and verify it does not forward to upstream when
served locally, mirroring the existing `/api/plate` test assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@desktop/broker.test.js`:
- Around line 162-174: The test 'a served read is DENIED when the token file is
missing/empty (fails closed)' verifies that invalid/missing tokens are denied
with 401 status, but it does not verify that the upstream server was never
contacted. Modify the upstream server creation to track whether it received any
requests (for example, by incrementing a counter or setting a flag in the
request handler), then add assertions after the two denial checks to verify that
the upstream server was never contacted, proving the broker failed closed
without forwarding to upstream.

In `@desktop/store.test.js`:
- Around line 21-30: The test claims to verify `id` ordering as a tie-breaker in
its name but the test data does not include any rows with equal priority and
created_at values, so the id sorting regression cannot be caught. Add additional
task fixtures to the seedTasks call that have matching priority and created_at
values but different id values, then update the corresponding assertion in the
store.plate call to verify that these tied tasks are correctly ordered by id in
ascending order. This will ensure that the id ASC tie-breaking logic is actually
exercised and validated in the test.

---

Nitpick comments:
In `@desktop/broker.test.js`:
- Around line 136-160: The test only validates `/api/plate` local serving and
token-gating behavior, but Phase 3 also serves `/api/queue` which lacks
verification here. Add test assertions to verify `/api/queue` follows the same
token-gate and local-serving pattern: first insert a queue record into the task
table in the test setup, then add requests to `/api/queue` without a token
(expecting 401), with a valid token (expecting 200 with the queue data from
store), and verify it does not forward to upstream when served locally,
mirroring the existing `/api/plate` test assertions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0fc20449-84f9-457d-98ab-482bc0982191

📥 Commits

Reviewing files that changed from the base of the PR and between 0e315f4 and 6e7183b.

📒 Files selected for processing (6)
  • desktop/broker.js
  • desktop/broker.test.js
  • desktop/main.js
  • desktop/package.json
  • desktop/store.js
  • desktop/store.test.js
🚧 Files skipped from review as they are similar to previous changes (3)
  • desktop/package.json
  • desktop/main.js
  • desktop/broker.js

Comment thread desktop/broker.test.js Outdated
Comment thread desktop/store.test.js
The broker now ANSWERS /api/plate and /api/queue itself, reading the SQLite work-state
(node:sqlite, built-in -- no native dep) and the queue files directly, instead of
forwarding them to FastAPI. It owns the token gate for what it serves (FastAPI's check
never runs on a broker-served response) and keeps the Host guard. Everything else still
forwards unchanged.

Scope: parity-verified against the live FastAPI (broker output == FastAPI output, byte
for byte). settings/procedures/pending are deferred to later increments -- the parity
check caught that /api/settings reads an environment-detected terminal and serializes a
float budget, so it isn't a pure static read yet. The liveness-bearing reads
(inflight/runs) and the SSE live mirror keep forwarding until the Phase 5 native layer
migrates the flock/pid liveness.

- desktop/store.js: plate() (SQLite) + queue() (files), each mirroring its Python reader.
- desktop/broker.js: a SERVED map routes those two; token gate + 401; rest forwards.
- tests: store unit tests + a broker served-vs-forwarded test (16 total). A live parity
  check against FastAPI confirms exact-match output.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant