fix(routing): select concrete free model groups - #971
Conversation
Exclude OpenRouter's aggregate free router while retaining discovered concrete free models. Group exact model identities across provider accounts and remove fixed inference deadlines, including readiness races. Signed-off-by: Seongho Bae <me@seonghobae.me>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueNote Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthrough모델 검색과 부트스트랩이 모델 그룹 및 fingerprint ID를 사용합니다. 공급자 호출은 선택적 타임아웃과 명시적 취소를 지원합니다. OpenRouter ZDR 라우팅, 임베딩 실패 기록, readiness 동시성, 레거시 에이전트 동기화 및 관련 회귀 검증을 갱신합니다. Changes모델 검색 및 부트스트랩
공급자 취소 및 readiness
ZDR 라우팅 및 임베딩
계약 및 운영 지원
Estimated code review effort: 5 (Critical) | ~120 minutes Merge Risk: 🔴 Critical · up to The current head still relies on temporary source-rewriting machinery instead of committed production fixes, while provider discovery can wait indefinitely or leak background work and embedding and failover paths retain correctness and privacy risks. These issues can cause hangs, resource exhaustion, incorrect routing, or loss of privacy guarantees, so the PR is not merge-ready and should remain blocked. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 52.20% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 341 functions across 44 files. (3 skipped: 3 unsupported.) ✨ Finishing Touches 💡 2📝 Generate docstrings 💡
⚔️ Resolve merge conflicts 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Routing contract: provider family is not a product grouping. Selection and measurement use |
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
Signed-off-by: Seongho Bae <me@seonghobae.me>
|
@jules Fresh exact-head review on This matters with the fleet's current measured consumer evidence: same-day model-backed Actions stalls have grown from roughly 649.5s to 1332.6s, 1462.9s, and 2161.9s while holding scarce job capacity. Do not 'fix' that by reintroducing a global model wall-clock timeout: #1053 correctly moves the default application model timeout to Please repair this same branch by normal descendant after re-reading live main/head. RED/GREEN acceptance:
Also update this PR's stale 'current exact head' prose after the causal repair; predecessor tests/reviews do not transfer. Keep Draft until the unchanged repaired head has exact tests/security/review/SBOM/provenance. |
|
Scheduled review-feedback autofix for this PR head.
|
|
Fresh fleet revalidation on current exact head The last exact-head hosted full suite is also a real source/test RED, not merely queue starvation: Tests run Required GREEN: first reconcile all 12 protected descendants with a normal non-force descendant while preserving this PR's valid routing/endpoint/bootstrap deltas; then re-run the seven concrete failures and repair any residual current-tree defects causally; only after that regenerate the full exact-head required protection set ( |
|
Scheduled review-feedback autofix for this PR head.
|
Preserve both histories with a normal two-parent merge against protected main@414f22973658c4ddc3d4320fcf7acd9b4e8ba991. Six overlapping files were conflict-free under exact merge-base 212ff43; main-only security, test, documentation, and removed legacy workflow deltas are retained. Repair the seven hosted test failures without weakening production fail-closed behavior: inherit main's json/shutil/subprocess admin-test imports, inject the existing deterministic authoritative embedding counter into four synthetic OpenRouter ZDR fixtures, and durably upsert each legacy bootstrap identity before restart. TaskOrchestrator construction is intentionally not a persistence side effect. No production token-count fallback, identity heuristic, timeout, retry, provider/model selection, or check threshold is changed.
The exact-head push run 34065942588 failed at workflow startup with zero jobs because the restack left one three-way conflict marker block around setup-node. Retain the protected-main setup-node v7 immutable pin and the branch's bounded non-model setup step. No model, gateway, routing, timeout, or permission behavior is changed.
The review thread identified a vacuous direct-check path: invoking either new identity regression file as documented for neighboring checks ran zero tests and returned success. Add pytest entry points to both bootstrap and catalog companions and register both commands in README. Production discovery, persistence, routing, and admission behavior is unchanged.
Exact-head Tests run 34066167599, job 101575237781 failed with 325 collection errors sharing one cause: contextual_orchestrator/orchestrator.py contained NUL bytes and was only 600,060 bytes after connector-side blob truncation. Restore the complete 792,692-byte conflict-free three-way merge result. No source semantics are otherwise changed.
RED: durable provider bootstrap currently alphabetizes enabled agent IDs instead of preserving the selector's cost/model-group order.
Keep resolved durable IDs in the selector's cost/model-group order while retaining set-based membership and identity-collision checks. Documents the exact external-review regression and acceptance state.
The ordered durable identity accumulator now owns report order; retain only the membership set used by collision and withdrawal checks.
|
Canonical-owner finding to integrate after the current exact-head writer completes (verified 2026-09-07): |
|
Scheduled review-feedback autofix for this PR head.
|
|
Fresh canonical-boundary review at exact head The current This head is not a complete #1106 repair: it still creates This is evidence/handoff only. The PR remains Draft with its existing unrelated unresolved findings and terminal CodeQL PR failure; no stale GREEN is promoted. |
|
Exact-head CodeQL RCA for This is not a product-source or scan-result failure in #971, and rerunning the leaf jobs cannot create the missing cross-repository authority. Canonical owner |
|
Exact-head verification for
|
|
Scheduled review-feedback autofix for this PR head.
|
|
Read-only consumer handoff from Protected fast-mlsirm currently has stale architecture/ADR language that assigns model-backed GitHub calls directly to A fresh owner check shows contextual-orchestrator currently exposes no GitHub Releases, so fast-mlsirm cannot truthfully replace the stale provider-key contract with a released immutable CO dependency yet. Please preserve #971's |
|
Scheduled review-feedback autofix for this PR head.
|
Current owner state
This remains the broad trusted owner lane for concrete
orchestrator/freegrouping, provider discovery/routing, endpoint lifecycle, durable bootstrap identity and related routing behavior. It is Draft and not merge-ready.414f22973658c4ddc3d4320fcf7acd9b4e8ba991838cbcb77218f6a87c9ec222b5a26b0c9066ec3f2026-09-07 durable selection-order repair
External thread
PRRT_kwDOTB3CTs6elTYCwas causal: durable bootstrap converted the selector result to a set and returned alphabetically sorted persisted IDs, while the ephemeral report preserved cost/model-group order. RED commit98aed1a811cb894881b4c9aeb20de4f0b00fb634adds a two-model durable-pool oracle whose selected order is deliberately opposite lexical agent-ID order. Successord469085df5cf6db4cf57cd6da43569f77920c463keeps set membership for fail-closed collision/withdrawal checks but separately records, activates, and reports resolved persisted IDs in selector order. It adds no ranking, provider quota, timeout, or dependency.Exact-head Security and Quality run 34068287994 is terminal GREEN: unchanged full suite/package quality, Hypothesis/Atheris fuzzing, CodeQL, dependency audit and SBOM all succeeded. Code Quality
34068286556is also GREEN. Remaining SAST, compatibility CodeQL, Security Scan, Noema, OpenCode and Strix lanes are current-head but still queued, so this PR remains Draft.Restack and hosted-RED repair
The previous head
df587e1704ecd12e355b4d1a70facb387a4648cbdiverged from 13 protected descendants. The successor preserves both histories. The six paths changed on both sides were merged with exact base212ff437dc297613289dba2e6064ade9942e07d8: five were conflict-free, while the setup-node block in.github/workflows/opencode-hourly-loop.ymlrequired the explicit resolution recorded below. All 29 main-only path deltas, including security/test/docs changes and deletion of obsoleteci.yml/fuzz.yml, were inherited from protected main.Tests run
33667736989, job100373515693had reported 7 failed, 3439 passed, 2 skipped. This successor repairs those failures without weakening production behavior:tests/test_admin_contract.pyinherits protected main'sjson,shutil, andsubprocessimports._ExactTestCounter; production remains fail-closed when authoritative token sizing is unavailable.sync_discovered_agentsbefore restart.TaskOrchestratorconstruction intentionally remains free of an implicit persistence side effect.Restack commit
8b05b831…initially produced push run34065942588with failure and zero jobs because one unresolved three-way marker remained aroundactions/setup-node;521e613…removed that artifact while preserving the immutable setup-node v7 pin and bounded non-model setup timeout.6ddc6b4…then closed the vacuous direct-check path by giving both bootstrap identity fixtures real pytest entry points and registering both in README. Its hosted Tests run34066167599, job101575237781exposed a single publication defect behind 325 collection errors:orchestrator.pyhad been truncated to 600,060 bytes and contained NUL bytes. Current successor1e59d4f…restores the complete 792,692-byte merged blob; the GitHub content API reports no NUL characters. Fresh SAST34066349130, CodeQL34066349112, merge scheduler34066348028, Strix34066348040, OpenCode34066348094, Noema34066348025, Security/Quality34066349029, Security Scan34066349074, and Code Quality34066347141runs materialized for this exact head. Code Quality is terminal GREEN; the Security/Quality CodeQL+supply-chain+SBOM job is terminal GREEN; its unchanged full-suite and coverage-guided fuzz jobs are still executing. The other exact-head lanes remain queued/pending. Predecessor results do not authorize this head.2026-09-07 ambiguous bootstrap-admission repair
External thread
PRRT_kwDOTB3CTs6eh3BQremained causal: both bootstrapselectors let lexical provider/model identity decide a bounded cutoff when
price evidence was equal or incomplete. Test-only RED
eeb9cc1bafe579032ab48778fa08c24e0b3f0aa1produced exact-head Security andQuality run
34071330949, job101589111271: the two new contracts failedwith
DID NOT RAISE, and the unchanged remainder was3490 passed, 2 skipped.GREEN implementation
575148b9df50b4e3840bc25f4fba1367d8f6a654retains explicitprovider/model-group availability constraints but rejects a selected/excluded
capacity boundary sharing the same comparable-cost or unknown-price state.
The error tells operators to supply comparable price evidence or include the
entire tied class. It adds no score, weight, quota, fuzzy identity, or learned
quality claim. The doctoring record now distinguishes direct provider
bootstrap's model-group contract from the discovery CLI's additional
provider-spread contract; ADR 0032 is demoted to Proposed while protected
delivery remains pending. Successor
50b0c869571f361865efd22c7c596fda978679e8adds the corresponding equal-known-price edge cases without changing production. Exact-head Security/Quality34072366669, SAST34072366670, CodeQL34072366652, and Security Scan34072366665have materialized but are not yet terminal. Both related review threads remain open until current-head focused/full evidence succeeds.2026-09-08 unmodeled diversity-decision repair
Review thread
PRRT_kwDOTB3CTs6eh3BQremained causal after theequal/incomplete-price repair: the two hand-authored diversity passes could
still let a distinct provider/model-group label displace a lower-cost
candidate. The same passes could also reorder a complete candidate pool,
making the label an undocumented routing utility.
RED fixtures first failed with
DID NOT RAISEfor both selectors at thebounded cutoff and complete-pool ordering boundaries. Successor
7206c5f6b62d654706961da793a70fca16ce03c8rejects bounded displacement;838cbcb77218f6a87c9ec222b5a26b0c9066ec3falso rejects full-poolreordering. Both compare the diversity proposal with the price-evidenced
candidate sequence and fail closed on any difference until an explicit
decision model supplies utility evidence. No provider quota, outage weight,
lexical tie-break, learned score, timeout, or fallback was added.
Exact-tree local verification is 133 passed across all direct selector
consumers, review-gateway integration, and CLI admission coverage; Python
compileall and
git diff --checkpass. The local runtime did not provideRuff, so fresh hosted lint/full-suite/security/model-review Checks remain
mandatory and predecessor GREEN does not transfer. Gap baseline, doctoring,
CLI help, and CHANGELOG are aligned to the Proposed/fail-closed boundary.
The PR stays Draft.
Preserved valid direction
The branch still owns valid repaired behavior that must survive review: purpose-complete self-modifying source-fix machinery stays absent; endpoint-race/process-exit cleanup contracts; bounded provider-embedding worker shutdown; durable bootstrap/catalog identity; provider discovery and free-pool evidence boundaries; default inference timeout
null; separation of user cancellation, provider termination, audited administrative timeout and infrastructure loss.GitHub Actions consumers request only
orchestrator/free. Provider/model/group/credential discovery, free/privacy/capability admission, serving order and failover remain contextual-orchestrator authority. OpenAI integration may remain globally available while OpenAI-derived candidates remain excluded from the free pool under the current policy. Private candidates require ZDR admission.Still-live owner RED
838cbcb77218f6a87c9ec222b5a26b0c9066ec3f; keep the separate migration thread open until exact-head hosted verification and consumer review prove it.Promotion contract
Before Ready or ordinary protected merge:
No force push, destructive rebase, stale-tree overlay, source-neutral retrigger, predecessor-evidence transfer, self-approval, administrator bypass or gate weakening is acceptance evidence.