fix(gateway): remove implicit model request timeout - #1053

sequenceDiagram
  participant Admin
  participant server
  participant TaskOrchestrator
  participant _AgentPoolStore
  Admin->>server: GET timeout_policy/history
  server->>TaskOrchestrator: 인증된 정책 조회
  TaskOrchestrator->_AgentPoolStore: 정책 및 revision 이력 요청
  _AgentPoolStore-->>TaskOrchestrator: 페이지 데이터 반환
  TaskOrchestrator-->>server: 정책 응답 생성
  server-->>Admin: 정책 및 history 반환

sequenceDiagram
  participant EmbeddingRequest
  participant server
  participant ProviderEmbeddingBatchBackend
  participant Provider
  EmbeddingRequest->>server: /v1/embeddings 요청
  server->>ProviderEmbeddingBatchBackend: timeout=None 또는 유한 timeout 전달
  ProviderEmbeddingBatchBackend->>Provider: 완료 대기
  Provider-->>ProviderEmbeddingBatchBackend: 임베딩 결과
  ProviderEmbeddingBatchBackend-->>server: 완료 상태와 결과 반환
  server-->>EmbeddingRequest: 임베딩 응답 반환

sequenceDiagram
  participant Provider
  participant TaskOrchestrator
  participant server
  participant Client
  Provider-->>TaskOrchestrator: TimeoutError 또는 ConnectionError
  TaskOrchestrator-->>server: ProviderUpstreamError(provider_outcome_unknown)
  server-->>Client: 502와 x-should-retry:false 반환

Merge Risk: 🟡 Moderate · up to 76c04

A local model admission timeout can unnecessarily fail a request instead of trying another candidate, and API consumers are not informed of a normal fail-closed history response. These should be fixed before merge.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

_{Comment @coderabbitai help to get the list of available commands.}

exact-head 재검증

• head: 60c04f982cfd354a8e46566dc437586c73e770a6
• base: f16bd35c887a2426cabad0a9f5958a4e770e6e71
• 관련 gateway/repair/embedding/race: 264 passed
• 전체 suite: 3396 passed, 2 skipped in 652.18s
• 중앙 .github/main@26cfc33 OpenCode·Strix·Noema timeout 계약: 70 passed, 4 subtests passed
• git diff --check: 통과
• 미해결 review threads: 0

현재 hosted required checks는 organization Actions 적체로 queued 상태입니다. 제품/gateway 변경이므로 admin bypass는 사용하지 않았고 squash auto-merge만 설정했습니다.

latest-main exact-head 재검증

• head: a60ed897c0c9298f8be846ac5b1124349eb13acd
• base: dddaa6ee6c9b156a69fee62e46c88c5d98a75c37
• 관련 gateway/repair/embedding/race: 264 passed
• git diff --check: 통과
• 미해결 review threads: 0

직전 exact head 60c04f982cfd354a8e46566dc437586c73e770a6의 전체 suite는 3396 passed, 2 skipped였습니다. 최신 base 병합 뒤 전체 suite에서 tests/test_atheris_requirements_marker.py 1건이 실패했으며, 동일 실패를 깨끗한 origin/main@dddaa6ee6c9b156a69fee62e46c88c5d98a75c37에서 재현했습니다. 새 base가 병합한 Atheris lock marker 결함으로 이 PR delta와 무관하며, 숨기거나 이 PR에 섞어 고치지 않습니다.

현재 보호 상태는 required checks와 독립 승인 대기 BLOCKED이며 squash auto-merge가 설정돼 있습니다. 제품/gateway 변경이므로 admin bypass는 사용하지 않습니다.

@coderabbitai review

@coderabbitai review

Current revision verification: 76c047585f54fcbe940fe168412f51627d3f79dd.

uv run --frozen pytest tests/test_model_timeout_policy.py -q passed 51 tests in 10.83 seconds in an isolated checkout of this revision. This is focused model-timeout policy regression evidence, not a full-suite, hosted approval, deployment, or buyer KPI result.

The missing CodeQL verdict was traced to central run 34122498232. Python job 101756437515 failed status publication with HTTP 403 for both target-app-token and github-token. Current opencode-agent installation 141441800 covers all repositories but grants statuses/actions read only; its anomalyco-owned registration also requests read only. A consumer rerun cannot add those permissions. Investigation and artifact provenance are proposed in #1107; existing repair ownership remains #1053 and the central workflow owner.

Additional independent live evidence: central .github run 34299034731/job 102308769876 completed failure on target head a34dc5af8363a86531d70e51983ad336b9f57096. Its trusted materialization ref is 7fd571dbcdbae6acf29d8f4ee704d7ba6297e4db, not the target PR head. Logs explicitly record vendoring and installing CO 414f229. Launcher blob e8c462abcc6b060cf47f173ff4c9e1f092cc6c78 and sidecar blob 38d9551a3273d9da8562efee1079f47acdfb819a match at both central refs. Both launcher ModelClient constructors omit timeout; pinned CO defaults to 90 and propagates self.timeout into transport. Uploaded artifact 10085227363 records Llama at 02:05:20.761 -> TimeoutError 02:06:50.877 UTC, then further Llama calls, and finally DeepSeek transient HTTPError at 02:07:37.330 followed by the caller 429. No correlation ID in these attempt lines proves they all belong to one logical request. Preflight candidate24/probed16/ready1/deferred8/rejected7 is not a one-model discovered pool: launcher intentionally returns ready plus demoted deferred agents, and target_ready8 is a fill target, not an admission minimum. This reinforces the existing timeout owner here; it does not prove a new 429 classification defect, zero internal failover, or that removing timeout recovers this run. Please preserve default-null and ambiguous-outcome replay safety together. Current-head contract coverage and the source/provenance gap are being reviewed before any additional code delta.

Independent exact-head validation at 76c0475: uv run --locked --group dev python -m pytest tests/test_model_timeout_policy.py tests/test_orchestrator_client_boundaries.py tests/test_provider_reliability.py tests/test_passthrough_provider_failover.py -q => 200 passed in 15.52s. Worktree clean and HEAD unchanged at completion. Initial use of the primary checkout virtualenv gave 199 passed/1 failure because openai was absent; this is resolved by this PR locked dev group (openai==2.54.0), not a runtime code change. Existing tests verify default None, selected-model numeric timeout propagation to the provider-open boundary, policy clear/restore/history, and no replay after ambiguous timeout/connection outcomes. This is local focused evidence, not a fresh full-suite/security/review approval or live recovery claim. No duplicate implementation was added.

Review reconciliation at exact head 76c0475: the last five inline findings were compared against current source and the independently rerun 200-test focused suite. (3944469324) ordinary pool writes are covered by concurrent timeout-preservation tests. (3944469333) test_timeout_changes_are_visible_in_operator_audit checks set/clear/restore committed revisions and no new audit entry for a stale rejected update. (3944469376) server._admin_purpose now recognizes timeout_policy/history as audit_replay; the HTTP regression asserts durable=True. (3945483292) _LocalProviderAdmissionTimeout and test_local_admission_timeout_preserves_send_boundary distinguish pre-send slot failure from post-send unknown outcome; the wrapped-admission test rejects inherited replay authority. (3945483293) locked dev declares openai==2.54.0 and the SDK test is no longer silently skipped in the validated environment. No additional duplicate fixes are warranted for these findings; this reconciliation is not an independent approval and does not auto-resolve reviewer threads. Remaining telemetry gap is distinct: the generated error request_id joins error response to error log, not all earlier provider_attempt lines, and cannot retrospectively correlate the historical incident.

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Fresh incident triage 2026-09-10, head 76c0475. The PR opening summary saying unresolved review threads = 0 is stale: five current unresolved threads were posted on 2026-09-09 at 04:52–04:53Z after that summary. They concern post-send timeout replay in chat/_invoke, one model execution deadline across streamed reads/admission, embeddings using the client-wide limit instead of selected-model policy, accepted timeout values overflowing socket clocks, and local pre-send admission failures misclassified by _invoke. These are substantive contract/safety findings, NOT a CodeQL dispatch/runner-only block.

Reviewed the complete 25-file delta and confirmed the reported boundaries are present: _open_model_provider resolves a socket timeout, server embeddings computes a deadline from orchestrator.client.timeout, ModelAgent accepts 2**63, and the dedicated admission handling is added only to passthrough classification. No bulk conflict resolution or bypass merge will override those findings. The current main 1e25c71 already contains the separately delivered default-null prerequisite and its full local Quality/fuzz/CodeQL run 34424893677 is successful; do not revert it while repairing this broader policy branch.

Todo: preserve this PR/history; add focused REDs for all five current threads; repair each at the owning boundary with no implicit global cap and no unsafe post-send replay; non-force integrate current main, preserving its provider metadata, cost accounting, pin, and request correlation deltas; rerun exact-head policy/SDK/stream/embedding/admission regressions before head-guarded merge. Infrastructure-only bypass remains explicitly authorized by the user, but does not waive these code findings. Other independently safe stalled PRs are being processed without waiting on this branch.

2026-09-12 lifecycle correction at exact head 76c047585f54fcbe940fe168412f51627d3f79dd: the PR is non-mergeable, CodeQL PR run 34098501126 is terminal failure, and five source-policy threads remain unresolved (post-send outcome-unknown replay, one end-to-end deadline, embedding model-policy authority, socket-representable finite values, and pre-send admission classification). Merged prerequisite #1118 carried only the separately verified nullable-default subset; it does not retire these valid remaining deltas. Keep this PR open and Draft pending non-force restack plus direct RED→GREEN repair.

Exact-head timeout follow-up receipt (Proposed; not merge authority)

• RED commit: a8a033afa003656bbe57f2ac12f6291aada9bbd9
• GREEN/current head: e79d6ed222fafe15fb4157cbadd33dcb13d9b0b6
• Exact tree: 3fee836d60264eb8d90847ecd8571861dac1f5b5
• Protected base: 012beaacd0631f8cd3391c77744eeb626269b5de (79 ahead / 0 behind, mergeable)

Repaired and regression-bound:

1. post-send timeout/reset is never replayed (model_timeout only for an explicit finite deadline; otherwise provider_outcome_unknown);
2. one monotonic deadline spans admission, connection, retries, and every stream read;
3. synchronous embeddings resolve each selected model's timeout;
4. admitted timeout values are capped at the socket-safe 2,147,483,647 seconds in runtime, persistence, and OpenAPI;
5. _LocalProviderAdmissionTimeout proves pre-send state and fails over directly without same-agent retry.

Verification on the exact tree:

• 3686 passed, 3 skipped, 1 deselected (the deselected case requires the unavailable released fast-mlsirm native artifact);
• Ruff, compileall, and git diff --check: clean;
• public-object docstrings: 100%;
• branch-aware aggregate coverage: 94% (therefore no 100% coverage claim).

The PR remains Draft/Proposed. Current hosted exact-head runs are queued/skipped by Draft policy; no merge, auto-merge, bypass, synthetic status, manual rerun, force update, or close was performed.

Lifecycle admission update for exact head e79d6ed222fafe15fb4157cbadd33dcb13d9b0b6.

The five substantive timeout/admission findings are now repaired with RED→GREEN coverage, the exact-tree receipt is recorded above, the branch is mergeable, and unresolved review threads remain zero. I therefore changed this PR from Draft to Ready for review. This is review/CI admission only, not approval or merge authority.

The Ready transition emitted fresh Security and Quality run 34697874302, currently queued. Existing exact-head CodeQL remains in progress, Security Scan remains queued, and SAST is successful. Do not transfer predecessor-head reviews or local verification into hosted GREEN.

OpenCode Review Overview

• Head SHA: e79d6ed222fafe15fb4157cbadd33dcb13d9b0b6
• Workflow run: 34698219615
• Workflow attempt: 1
• Gate result: COVERAGE_BLOCKED
• Coverage gate: failure
• Model pool: unknown
• Verdict: COVERAGE_BLOCKED
• Formal review: fix(gateway): remove implicit model request timeout #1053 (review)

Coverage evidence did not pass, so approval is blocked. The formal pull-request review is the source-backed diff review, not this status comment.

Exact-head hosted RED and lifecycle correction for e79d6ed222fafe15fb4157cbadd33dcb13d9b0b6 (2026-09-12 UTC).

• Security and Quality run 34697874302, job 103564380561: 1 failed, 3687 passed, 2 skipped.
• Failing assertion: tests/test_spend_analytics.py::test_exact_output_without_prompt_usage_is_explicitly_unavailable expected tokenizer but observed mixed.
• RCA: the restack retained the tokenizer-only assertion but lost the earlier explicit isolation of the optional fast-mlsirm judge. With the full dependency set, the judge contributes a second usage source, so the fixture no longer measures only the contract named by the test.
• Minimal repair prepared from parent e79d6ed2: patch _resolve_fast_mlsirm_components to None only inside this fixture, preserving production judge/spend behavior; update CHANGELOG and the Proposed gap baseline. Focused spend suite passes 4 passed; compileall and diff-check are clean. Local commit 5772c20b8005988847f8a81d45d30eacc5c554ff is not remote evidence.

The execution environment rejected the non-force push as unauthorized repository-content egress. I did not bypass that restriction through another write path and did not rerun the failing head. Because the repair is not on the remote branch, I returned the PR to Draft. CodeQL run 34697468789 is also first-pass pending/bootstrap failure and is not GREEN.