Skip to content

Desktop: Phase 3 (step 2) broker serves /api/procedures - #71

Merged
zkann merged 1 commit into
mainfrom
desktop-broker-reads2
Jun 17, 2026
Merged

Desktop: Phase 3 (step 2) broker serves /api/procedures#71
zkann merged 1 commit into
mainfrom
desktop-broker-reads2

Conversation

@zkann

@zkann zkann commented Jun 17, 2026

Copy link
Copy Markdown
Owner

Continues Phase 3: the broker now answers /api/procedures itself, alongside /api/plate + /api/queue.

What moved

desktop/store.js ports three Python helpers so the broker can build the procedure list directly from the SOP files + runs.jsonl:

  • iter_sops skip rules (rglob *.md, skip runtime dirs / INDEX.md / _template.md / DIGEST.md / dotfiles),
  • autonomy_level_from_meta (explicit field → else interactive→with_me → else status→on_its_own/prepare_ask),
  • _cost_estimates per-SOP median of prior ok runs (numeric sort, even/odd median, non-ok/bool/negative excluded).

Each row: id, title, draft, interactive, needs_inputs, cost {estimate,n}|null, autonomy, sorted by title.

Parity

Verified by semantic parity against the live FastAPI (parsed-JSON deep-equal). A whole-dollar cost serializes as 1 here vs 1.0 in Python — identical after JSON.parse (what the SPA does), so it's not a real mismatch; the check compares parsed values, not bytes. plate + queue stay parity-clean.

Still forwarding

settings (reads an environment-detected terminal, not a pure static read), pending, and the liveness-bearing reads + the SSE live mirror (the Phase 5 flock/pid migration).

Tests

node --test: 19 (added a procedures unit test: autonomy derivation, cost median, INDEX skip, title sort) + the live semantic parity check.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added a Procedures view that surfaces procedure metadata (title, draft/interactive flags, autonomy level, and cost estimates) using available historical run data.
    • Improved local API support by adding a dedicated procedures endpoint for faster, broker-gated reads.
  • Bug Fixes
    • Procedure ingestion now skips non‑UTF-8 markdown files to avoid malformed content breaking the procedures list.
  • Tests
    • Expanded coverage for metadata parsing, autonomy/cost derivation, sorting, tie behavior, and UTF-8 skipping.

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5e15114b-f314-4f77-903b-21cc657a9e6b

📥 Commits

Reviewing files that changed from the base of the PR and between e14d998 and e1c3b2e.

📒 Files selected for processing (3)
  • desktop/broker.js
  • desktop/store.js
  • desktop/store.test.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • desktop/broker.js

📝 Walkthrough

Walkthrough

desktop/store.js gains a new procedures(sopDir) function that scans SOP Markdown files, parses frontmatter, derives autonomy, and computes median cost from runs.jsonl with UTF-8 safety. The function is exported and wired into desktop/broker.js as a token-gated GET /api/procedures route. Tests for the new function are added in desktop/store.test.js.

Changes

Procedures Endpoint

Layer / File(s) Summary
Store procedures implementation with UTF-8 safety
desktop/store.js
Introduces readTextStrict() for strict UTF-8 file decoding, implements procedures(sopDir) which recursively collects SOP .md files (skipping INDEX.md and reserved directories), parses frontmatter to derive autonomy, reads runs.jsonl to compute median cost_usd from ok runs only (or cost: null for drafts/no runs), and returns sorted procedure objects. Exports the function from the module.
Store procedures tests
desktop/store.test.js
Validates metadata derivation from frontmatter, title-based sorting, autonomy field mapping, median cost computation vs. null defaults, deterministic ordering of identically-titled procedures, and UTF-8 validation parity with queue().
Broker /api/procedures route
desktop/broker.js
Extends the SERVED GET routing table with /api/procedures, returning { procedures: store.procedures(sopDir) } behind the same token check as existing broker-served endpoints.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • zkann/smbos#47: Implements the same /api/procedures SOP catalog concept via scripts/dashboard_app.py scanning SOP markdown frontmatter, directly paralleling this PR's store.js+broker.js approach.
  • zkann/smbos#69: Introduced the desktop/broker.js reverse-proxy routing facade that this PR's new /api/procedures route is built on top of.
  • zkann/smbos#70: Established the token-gated broker static-read pattern (/api/plate, /api/queue via store.js) that this PR extends with the /api/procedures endpoint.

Poem

🐇 Hopping through the SOP tree,
Skipping INDEX, wild and free,
Frontmatter parsed, autonomy found,
Median costs computed, all sound—
/api/procedures now replies,
With sorted wisdom, carrots, and ties! 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: implementing broker service for /api/procedures endpoint as part of Phase 3 migration, which directly aligns with the core functionality added across all three modified files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch desktop-broker-reads2

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e14d998394

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread desktop/store.js Outdated
const out = []
for (const p of iterSops(sopDir)) {
let m
try { m = parseFrontmatter(fs.readFileSync(p, 'utf8')) } catch (_) { continue } // skip unreadable SOP

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip invalid UTF-8 SOPs

When a SOP file contains invalid UTF-8 bytes, fs.readFileSync(p, 'utf8') silently replaces them with , so this catch never runs. The FastAPI implementation this replaces catches UnicodeDecodeError and skips that SOP (covered by test_procedures_skips_unreadable_sop), so after the broker starts serving /api/procedures itself a corrupt/non-UTF-8 .md with frontmatter is returned to the UI instead of being omitted, breaking endpoint parity and exposing an unusable procedure.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, fixed. Node fs.readFileSync(p,"utf8") substitutes U+FFFD on invalid bytes (no throw), so a corrupt SOP/queue file would be parsed garbled and INCLUDED, where Python read_text(encoding="utf-8") raises and _procedures/_queue skip it. Added readTextStrict() using TextDecoder({fatal:true}) so the read throws and the caller's try/catch skips, matching Python. Test added (a non-UTF-8 ops/ and queue/ file are skipped).

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@desktop/store.js`:
- Around line 54-71: The collectMd function currently traverses into all
directories and only filters out SKIP_DIRS later in iterSops, causing
unnecessary traversal of large directories like archive and active-runs. Modify
collectMd to accept SKIP_DIRS as a parameter and check if each directory is in
SKIP_DIRS before recursing into it. Update the recursive call in collectMd to
pass SKIP_DIRS along, and update the call to collectMd in iterSops to pass
SKIP_DIRS as the third argument. This will prevent traversal into excluded
directories during collection rather than filtering them after collection is
complete.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f52d440e-a85b-4e3a-a2ab-d775b2dacee3

📥 Commits

Reviewing files that changed from the base of the PR and between fd3af21 and e14d998.

📒 Files selected for processing (3)
  • desktop/broker.js
  • desktop/store.js
  • desktop/store.test.js

Comment thread desktop/store.js Outdated
The broker now also answers /api/procedures from the SOP files + runs.jsonl directly:
the procedure list with each SOP's derived autonomy level, cost estimate (median of
prior 'ok' runs), and the draft/interactive/needs_inputs flags -- porting iter_sops'
skip rules, autonomy_level_from_meta, and _cost_estimates' per-SOP median into store.js.

Verified by SEMANTIC parity against the live FastAPI (parsed JSON deep-equal, so a
whole-dollar cost serialized as 1 here vs 1.0 in Python -- identical after JSON.parse,
which is what the SPA does -- doesn't read as a mismatch). plate + queue still parity-clean.

Still forwarding: settings (reads an environment-detected terminal), pending, and the
liveness-bearing reads + SSE (Phase 5 flock/pid migration).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@zkann
zkann force-pushed the desktop-broker-reads2 branch from e14d998 to e1c3b2e Compare June 17, 2026 23:28
@zkann

zkann commented Jun 17, 2026

Copy link
Copy Markdown
Owner Author

Adversarial self-review hardening (folded into the latest push), in addition to the bot fixes above:

  • [P2] iter_sops sort order: Python sorted(rglob) compares path COMPONENTS; a JS full-string sort orders "/" vs "-" differently, so a nested SOP vs a prefix-sharing sibling (clients/x.md vs clients-flat.md) with the SAME title would tie-break in the opposite order. Now sorts by component tuple. Test added.
  • [P3] median rounding: JS Math.round is half-up vs Python round() half-to-even; differ by $0.0001 only on an exact-half median (rare, sub-cent, UI shows cents). Documented as accepted tolerance rather than chasing banker's rounding.
    21 node --test cases; live semantic parity still exact (incl. a nested SOP).

@zkann
zkann merged commit c1531d9 into main Jun 17, 2026
7 checks passed
@zkann
zkann deleted the desktop-broker-reads2 branch June 17, 2026 23:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant