Repository navigation
[Interpretation Gap] Complete evidence-bounded contextual-orchestrator interpretation and verification #176
Description
Buyer problem
TEPP must turn validated numerical/event artifacts into explanations that a buyer can inspect and act on. Current work includes a deterministic orchestration router on protected main, interpretation-grounding gates in #69, and hourly workflow routing in #165. It does not yet execute a production interpreter→verifier workflow through contextual-orchestrator, prove that claims are supported by TEPP evidence, or quantify when deeper multi-agent compute outperforms direct routing.
Product outcome
Implement a versioned interpretation_gateway that consumes bounded TEPP evidence bundles and returns reviewed claims without becoming scientific authority.
Required workflow modes:
direct
verify
committee
conductor
abstain
The router allocates test-time compute using workflow stage, decomposition, recursion depth, access list, evidence ambiguity/risk, role-specific reasoning effort, and budget. Speed is not the primary objective; calibrated evidence support is.
Existing related work
- Protected
maincontains the deterministic adaptive router from PR feat(api): adaptive orchestration router selects modes under budget #47. - feat(interpretation): keep LLM claims hypothetical and evidence-cited #69 provides interpretation grounding/refusal metrics but not live execution.
- feat: route hourly product agents through contextual orchestrator #165 routes an autonomous-development workflow through contextual-orchestrator; that workflow is not the TEPP product interpretation port.
- feat(api): serve naruon POSTs on loopback with a live deadline #107/feat(api): admit LineageWeave as a modular analysis-run consumer #155/feat(api): publish completed analysis-run result contract #157/feat(api): expose temporal evidence context for LineageWeave Ask #158/feat(api): project LineageWeave evidence into cutoff-safe histories #159 define external analysis/evidence/result boundaries that interpretation must not confuse with web-verified fact.
Acceptance criteria
- Input is a versioned bounded evidence bundle containing only approved topic/event/network/psychometric summaries, uncertainty, model/config/cutoff identities, and allowed evidence references.
- Source documents, web pages, LLM output, and connector payloads are untrusted; embedded instructions cannot change system policy, tools, access lists, credentials, model authority, or merge/release state.
- Every claim cites supporting evidence IDs; unsupported or contradictory claims are removed, refused, or marked unresolved.
- The verifier checks direction reversal, causal overclaim, group generalization, missing uncertainty, incompatible run mixing, stale cutoff, source/result identity mismatch, and unsupported action recommendations.
- Topic, event, cluster, factor, path, and forecast labels remain interpretations; they cannot change numerical identities, membership, event promotion, or scientific claim status.
- Direct and deeper orchestration modes are evaluated at comparable budgets across decomposition, recursion, stages, access lists, and role-specific reasoning effort.
- Ablations report evidence precision/recall, unsupported-claim rate, contradiction rate, calibration, human/expert agreement, inter-model disagreement, abstention utility, cost/token use, and repeated-run variance.
- High disagreement, insufficient evidence, failed scientific gate, or policy denial produces
abstain/human-review rather than a persuasive completion. - Provider/model discovery and fallback are automatic through contextual-orchestrator; TEPP product code does not embed provider-specific secrets or model names as immutable scientific meaning.
- Provider terms, retention, region, confidentiality, and data-classification policy are checked before a model becomes eligible.
- Model/provider/prompt/schema/reasoning/workflow/access-list versions, token/cost records, evidence hash, and verifier outcome are recorded without prompt/source-text leakage into ordinary telemetry.
- Live contract tests exercise all configured provider families through the approved secret boundary while offline deterministic tests remain reproducible.
- Production statement/branch coverage and public docs remain 100%; prompt-injection, oversized output, malformed schema, unsupported field, retry/fallback, provider outage, and disagreement tests pass.
Product proof
Provide an anonymized TEPP result containing topics, event links, a network cluster, longitudinal effects, and uncertainty. Show that:
- a direct model produces a baseline interpretation;
- deeper orchestration is invoked only when the deterministic router selects it;
- the verifier rejects seeded unsupported/causal/direction-reversed claims;
- the final result cites exact allowed evidence and preserves uncertainty;
- an under-supported case abstains.
Scope boundary
LLM agreement cannot promote design or implementation maturity, establish construct validity, create observed event facts, or authorize high-stakes action. Contextual-orchestrator is a replaceable execution plane behind TEPP's versioned interpretation port.
Authority
Use ADR 0010, LLM_ORCHESTRATION, PRD §12, Privacy Governance, and the Fugu/Conductor/TRINITY and AI-risk sources recorded in the APA 7 research register.
Activity