Skip to content

docs(gap-baseline): record SDK-parity (33-37) scoping, reclassify item 32 #1746

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
seonghobae wants to merge 3 commits into
base: main
Choose a base branch
from
Open

docs(gap-baseline): record SDK-parity (33-37) scoping, reclassify item 32 #1746

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
dde453d
docs(gap-baseline): record SDK-parity (33-37) scoping, reclassify ite…
seonghobae Sep 2, 2026
208d8b7
docs(gap-baseline): correct item-32 batch-scope over-claim, pin SHA, …
claude Sep 2, 2026
7fbbbc1
Merge remote-tracking branch 'origin/main' into HEAD
seonghobae Sep 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/product-technical-gap-baseline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2627,3 +2627,27 @@ Higgins, S. S., Crepalde, N., & Fernandes, L. (2021). Segmented multiplexity: A
**900-second clarification.** The historical `NoemaRepairDeadlineExceeded` from the html4tree incident came from the retired caller repair path. The three literal `timeout --kill-after=20 900` invocations still present in `opencode-review-dispatch.yml` are separate containment limits for untrusted test-measurement commands; they are not model or Noema inference timeouts. Telemetry and runbooks must report the command class and phase separately.

**Evidence / acceptance.** Permanent tests forbid retry/deadline/sampling symbols in the caller and prove one gateway request, one attempt annotation, control-character-safe telemetry, missing-value rejection, valid trailing-comma normalization, and exact changed-line guidance. Fresh exact-head repository checks and reviews remain the admission authority; predecessor-head evidence is not transferable. The remaining runtime work is to preserve distinct `request_too_large`, discovery, rate-limit, provider transport, malformed-output, stale-head, and sandbox-command-timeout categories in hosted logs.

## Backlog items 33-37 (OpenAI SDK parity) scoping, and item 32 (batch scope) reclassification — 2026-09-02

**Status:** Scoping only, recorded per this session's throttle agreement. Implementation is deliberately deferred — opening new feature PRs right now would add to an already-saturated review queue (2154 queued / 2-4 in-progress GitHub Actions runs at the time of this entry, unchanged or worse than the 60-concurrent-runner-slot ceiling already documented elsewhere in this file) rather than clear it. Resume only once that queue meaningfully eases, or the repository owner explicitly directs otherwise — not on this session's own initiative.

**Method.** Read `contextual-orchestrator/contextual_orchestrator/server.py` (8744 lines, a raw `http.server`-based OpenAI-compatible gateway, not a framework router) directly from protected `main` (commit `212ff437dc297613289dba2e6064ade9942e07d8`), plus `batch_routing.py` and `file_registry.py`, and grepped for every major OpenAI SDK endpoint family's path string.

**Already implemented** (broader than expected): `/v1/chat/completions`, `/v1/responses`, `/v1/completions` (legacy, redirects to chat/completions), `/v1/embeddings`, `/v1/models`, `/v1/files`, `/v1/batch/embeddings`, `/v1/videos`, `/v1/images/generations`, `/v1/audio/{speech,transcriptions,generations}`, and `/v1/rerank` (a non-standard extension, not in the OpenAI SDK).

**Genuinely missing** (item 34 candidates — in the SDK, absent here):
- `/v1/moderations`
- `/v1/fine_tuning/jobs`
- `/v1/audio/translations`
- `/v1/images/edits`, `/v1/images/variations`
- `/v1/uploads`
- `/v1/realtime` (WebSocket, persistent-connection architecture — genuinely different shape from every other endpoint here; deserves its own ADR before any implementation attempt, not a same-pattern addition)

**Missing but likely low priority:** `/v1/assistants`, `/v1/threads`, `/v1/runs`, `/v1/vector_stores` (the Assistants API). OpenAI itself is sunsetting this family in favor of the Responses API, which this gateway already implements (`/v1/responses`). Building parity for a family the SDK's own vendor is deprecating is in tension with item 34's "if the SDK has it, implement it" principle — recorded as a judgment call to deprioritize, not a decision to never build it.

**Item 32 (batch endpoint scope) reclassified — corrected per Devin review, item stays open, not "policy already covers it."** `batch_routing.py` does implement a `RoutingPolicy` (`cost_router.py:106`, `self.policy = routing_policy or RoutingPolicy(...)`) that `CostRoutingCoordinator` uses to decide sync-vs-batch for *ordinary* `/v1/chat/completions`-shaped calls. But re-reading `server.py` and `cost_router.py` directly (at `main` commit `212ff437dc297613289dba2e6064ade9942e07d8`) shows this does **not** substantiate the original claim: the explicit `/api/v1/batch_routing_jobs` endpoint (`server.py:7443`) calls `coordinator.submit_batch(...)` (`:7453`) directly — `submit_batch` (`cost_router.py:970`) resolves the request's target model via `_resolve_batch_request` (only for ZDR-tagged requests) and hands off to `self.batch_backend.submit(...)`; neither path consults `self.policy`/`RoutingPolicy` at all, so nothing restricts this endpoint to batch-capable models. Separately, `CostRoutingCoordinator.__init__` (`cost_router.py:117-127`) defaults `self.batch_backend` to `LocalBatchBackend` unless a caller explicitly injects one, and `server.py`'s only constructor call (`server.py:5487`, `CostRoutingCoordinator(orchestrator)`) does not inject one — `PgLlmBatchBackend` (the class that wraps `pg_llm_batch.BatchAPIClient`) is never instantiated anywhere outside `tests/`. So "dispatching batch-eligible requests to pg-llm-batch through an injected BatchAPIClient" does not describe the server's actual default wiring. Item 32 remains open pending real verification of `/api/v1/batch_routing_jobs`'s actual model-resolution and backend wiring — not resolved by `RoutingPolicy`'s existence for a different code path.

@devin-ai-integration devin-ai-integration Bot Sep 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Batch classification matches pinned source

At the pinned revision, submit_batch bypasses RoutingPolicy; both production constructors leave batch_backend unset. Only tests instantiate PgLlmBatchBackend, supporting the open-gap classification.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


**Item 36 (S3-backed Files API) confirmed as a real gap.** `file_registry.py`'s current `FileOwner`/file-registry logic is a *provider-affinity* registry — it tracks which upstream provider a given gateway file id was proxied to, not a self-hosted store. No S3 (or other object-storage) integration exists. Item 36's "Files API can be self-implemented via S3" is not yet built.

@devin-ai-integration devin-ai-integration Bot Sep 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Files gap matches current behavior

The Files API proxies uploads upstream and persists only ownership metadata. No object payload store or S3 integration exists in the inspected implementation.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


**Suggested implementation order, if/when resumed** (smallest and most independent first, one PR each, not bundled): `/v1/moderations` → `/v1/audio/translations` (extends the existing audio family) → `/v1/images/{edits,variations}` → item 36's S3-backed Files store (infrastructure-shaped, kept separate) → `/v1/uploads` → `/v1/fine_tuning/jobs` (needs persistent job state, more complex) → `/v1/realtime` last, gated behind its own ADR given the architectural shift to a persistent connection model. Item 22 (Keyverse) was scoped in parallel and found mostly already covered by `ContextualWisdomLab/keyverse#103`'s ADR-accepted draft (ABAC/RBAC PDP + KV + credential store); see that item's own gap-baseline entry for detail.
Loading