diff --git a/AGENTS.md b/AGENTS.md index 9104dd1f4..eeb2c71ab 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -65,6 +65,35 @@ in this repo. add further `os.getenv` secret reads, and migrate toward the KV pattern as it is adopted. +### Noema LLM routing (orchestrator-only) + +- **Noema's LLM path is contextual-orchestrator only.** + `services.noema_agent:run_noema_agent` must not call + `resolve_runtime_llm_provider`, must not pick a tenant `OpenAIChatModel` / + `gpt-4o`, and must not copy draft email-writing clients that require a + tenant `model_profile_id`. Send the single model alias + `contextual-orchestrator` to a dedicated gateway inference token + HTTPS + `/v1` base URL from the Fernet tenant KV (`noema_orchestrator_token`, + `noema_orchestrator_base_url`). Catalog files + (`registered_agents.json` `provider_source=contextual-orchestrator`, + `task_agent_mapping.json`) stay catalog-only; do not wire Decision Points + or `mail.triage` dispatchers in the same slice. +- **Do not sequentially fail over** to the next agent or model inside naruon + or Noema. Do not copy OpenCode sidecar model lists. Never use + `COPILOT_GITHUB_TOKEN` or GitHub Models for Noema. +- **Upstream provider keys stay in the orchestrator KV**, not in naruon at + request time: `NVIDIA_NIM_API_KEY`, `NVIDIA_NIM_API_KEY_SUB`, + `BYTEZ_API_KEY`, `OPENROUTER_API_KEY`, `OPENAI_API_KEY`. List prices for + free-but-priced models are stored in the orchestrator, not here. Do not + reimplement the orchestrator catalog in this repo. Keep the existing + owner-scoped tools and opt-in writeback surface. + +- Noema gateway setup uses the signed `GET`/`PUT /api/noema-gateway` route. + It is scoped to the authenticated `(user_id, organization_id)` pair, stores + the token through `EncryptedString`, returns only `has_token` readiness, and + records generic audit events. Do not add target-user or mailbox credential + fields to this route without a separate membership/delegation ADR. + ### This repo's role in the ecosystem - **This repo (naruon) is the ECOSYSTEM HUB:** email/PIM that DOM-decomposes diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 9d2cbba18..80fb66da8 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -178,6 +178,19 @@ syntax, no userinfo/query or fragment, exact host membership in Missing allowlist configuration fails closed; the default provider path should leave `base_url` unset. +## Noema LLM routing + +Noema remains the in-process general agent (`registered_agents.json` → +`services.noema_agent:run_noema_agent`) with the existing owner-scoped +tools and opt-in writeback surface. Its LLM calls go only to +**contextual-orchestrator**: dedicated Fernet-KV inference token, HTTPS +base URL ending in `/v1`, and the single model alias +`contextual-orchestrator`. Catalog mappings (`mail.triage`, and the rest) +are catalog-only; they are not a live Decision Points dispatcher. naruon +does not hold upstream provider keys at request time and does not +sequentially fail over models. Design: +[`docs/architecture/noema-decision-agent.md`](docs/architecture/noema-decision-agent.md). + ## Batch embedding routing boundary Bulk, latency-tolerant embedding work (email import, backfills) does not call a diff --git a/CHANGELOG.md b/CHANGELOG.md index 7ec84c36f..0a6ff3d67 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,21 @@ ## [Unreleased] +- **Noema gateway setup:** added signed-session `GET`/`PUT /api/noema-gateway` + settings with HTTPS `/v1` allowlist validation, Fernet-backed token storage, + masked readiness responses, and generic audit records. The route keeps the + existing per-user organization scope and does not expose gateway tokens. + Doctoring records the OWASP ASVS 5.0.0 and NIST SP 800-63B-4 evidence mapping. +- **Noema LLM routing through contextual-orchestrator.** + `run_noema_agent` no longer calls `resolve_runtime_llm_provider` or a + tenant `gpt-4o` chat model. Completions go to the orchestrator gateway + (dedicated Fernet-KV inference token `noema_orchestrator_token`, HTTPS + `/v1` URL `noema_orchestrator_base_url`, model alias + `contextual-orchestrator`). Catalog `provider_source` is + `contextual-orchestrator`. Existing tools, owner-scope, and opt-in + writeback stay. This slice does not add a Decision Points / `mail.triage` + dispatcher. naruon does not sequentially fail over models and does not + read upstream provider keys (`NVIDIA_NIM_API_KEY`, `BYTEZ_API_KEY`, + `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, `COPILOT_GITHUB_TOKEN`) at + request time. See `docs/architecture/noema-decision-agent.md`. - 긴 이메일·첨부 본문을 의미 단위 청크로 임베딩한 뒤 기존 email/attachment 벡터 계약으로 평균화하고, 청크 요청·벡터 누적을 제한된 창으로 처리합니다. OpenAI `text-embedding-3-*`에는 저장 차원(`1536`)을 직접 요청하도록 보강했습니다. 합성 메일 fixture 5건(70청크)과 provider 요청 계약으로 1,536차원 벡터 경로를 검증했으며, 실행 시 선택한 임베딩 제공자에 본문·파싱된 첨부 텍스트를 전송할 수 있습니다. 회사 기밀 데이터는 fixture·commit·PR·log에 포함하지 않습니다. - EmailDetail 테스트가 지원하지 않는 스레드 병합/분리 버튼을 `textContent`뿐 아니라 `aria-label`과 `title` 접근 가능 이름으로도 검출하도록 바꿔, 아이콘 전용 버튼 회귀를 놓치지 않습니다. diff --git a/CLAUDE.md b/CLAUDE.md index be67bc80c..70c6b3119 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -110,6 +110,7 @@ indexes, and auditable writeback intent. `ARCHITECTURE.md` and ``` Next.js frontend ──> FastAPI backend (control plane) ──> Postgres + pgvector │ + ├──> Noema agent LLM ──> contextual-orchestrator (/v1) ├──> OpenAI-compatible LLM providers (Ollama locally) └──> outbound-only self-hosted connector (connector/) └──> customer IMAP/POP3/SMTP + CalDAV/CardDAV/WebDAV @@ -121,6 +122,12 @@ Next.js frontend ──> FastAPI backend (control plane) ──> Postgres + pgve (`services/threading_service.py` is the only thread-id assignment owner), vector search, AI summaries, ticket tasks, and server-authoritative calendar/WebDAV writeback intents. Authorization is deny-first RBAC + ABAC. + In-process **Noema** (`services/noema_agent.py`) keeps the existing + owner-scoped tool surface; its LLM calls go only to + contextual-orchestrator (model alias `contextual-orchestrator`, dedicated + gateway token + HTTPS `/v1` URL from the Fernet KV). naruon does not hold + upstream provider keys, pick tenant `gpt-4o`, or sequentially fail over + models. Catalog mappings are not a live dispatcher. - `frontend/` — Next.js workspace shell (Today dashboard, Mail, Calendar, Tasks, Projects, Context Search, AI Hub, Data, Security, Settings). Browser writes go through the same-origin `/api/*` proxy, which converts the HttpOnly diff --git a/backend/alembic/versions/0018_noema_orch_gateway.py b/backend/alembic/versions/0018_noema_orch_gateway.py new file mode 100644 index 000000000..fddcbdc6e --- /dev/null +++ b/backend/alembic/versions/0018_noema_orch_gateway.py @@ -0,0 +1,54 @@ +"""add noema contextual-orchestrator gateway columns + +Revision ID: 0018_noema_orch_gateway +Revises: 0017_merge_newsdom_carddav_heads +Create Date: 2026-08-16 00:00:00.000000 + +Noema judgments call only the contextual-orchestrator OpenAI-compatible +gateway. The dedicated inference token and HTTPS ``/v1`` base URL live on +``tenant_configs`` in the Fernet KV (token is EncryptedString). naruon does +not store upstream provider keys for this path. +""" + +from alembic import op +import sqlalchemy as sa + +revision = "0018_noema_orch_gateway" +down_revision = "0017_merge_newsdom_carddav_heads" +branch_labels = None +depends_on = None + +_TENANT_TABLE = "tenant_configs" + + +def upgrade() -> None: + connection = op.get_bind() + inspector = sa.inspect(connection) + if not inspector.has_table(_TENANT_TABLE): + return + for column in _noema_gateway_columns(): + if not _has_column(inspector, _TENANT_TABLE, column.name): + op.add_column(_TENANT_TABLE, column) + + +def downgrade() -> None: + connection = op.get_bind() + inspector = sa.inspect(connection) + if not inspector.has_table(_TENANT_TABLE): + return + for column in reversed(_noema_gateway_columns()): + if _has_column(inspector, _TENANT_TABLE, column.name): + op.drop_column(_TENANT_TABLE, column.name) + + +def _noema_gateway_columns() -> list["sa.Column"]: + return [ + sa.Column("noema_orchestrator_base_url", sa.String(), nullable=True), + sa.Column("noema_orchestrator_token", sa.String(), nullable=True), + ] + + +def _has_column(inspector, table_name: str, column_name: str) -> bool: + return any( + column["name"] == column_name for column in inspector.get_columns(table_name) + ) diff --git a/backend/api/noema_config.py b/backend/api/noema_config.py new file mode 100644 index 000000000..c84857b4e --- /dev/null +++ b/backend/api/noema_config.py @@ -0,0 +1,191 @@ +"""Signed-session settings for the per-user Noema gateway credential.""" + +from __future__ import annotations + +import hashlib + +from fastapi import APIRouter, Depends, HTTPException +from pydantic import BaseModel, ConfigDict +from sqlalchemy.exc import StatementError +from sqlalchemy.ext.asyncio import AsyncSession + +from api.auth import AuthContext, get_auth_context +from core.runtime_secrets import EncryptionConfigurationError +from db.models import AuditLog, SecurityAuditEvent, TenantConfig +from db.session import get_db +from services.llm_provider_urls import validate_llm_provider_base_url_async +from services.orchestrator_gateway import validate_orchestrator_gateway_url +from services.tenant_config_scope import ( + get_scoped_tenant_config, + new_scoped_tenant_config, +) + +router = APIRouter(prefix="/api/noema-gateway", tags=["noema-gateway"]) + + +class NoemaGatewayUpdate(BaseModel): + """Optional values for the signed-session user's Noema gateway.""" + + model_config = ConfigDict(extra="forbid") + + base_url: str | None = None + token: str | None = None + + +class NoemaGatewayResponse(BaseModel): + """Safe gateway state that never returns the Fernet-protected token.""" + + base_url: str | None = None + configured: bool = False + has_token: bool = False + + +def _resource_uid(auth_context: AuthContext) -> str: + """Return a stable, non-secret audit identifier for the scoped setting.""" + scope = f"{auth_context.organization_id or ''}:{auth_context.user_id}" + digest = hashlib.sha256(scope.encode("utf-8")).hexdigest()[:16] + return f"noema_gateway:{digest}" + + +async def _validated_base_url(value: str) -> str: + """Validate the HTTPS /v1 shape and the configured global-host policy.""" + try: + shaped_url = validate_orchestrator_gateway_url(value) + normalized_url = await validate_llm_provider_base_url_async(shaped_url) + if not normalized_url: + raise ValueError("gateway host is not allowlisted") + return validate_orchestrator_gateway_url(normalized_url) + except ValueError as exc: + raise HTTPException( + status_code=422, + detail="Noema gateway base URL is not allowed", + ) from exc + + +def _clean_token(value: str | None) -> str | None: + """Normalize a submitted token without recording or returning its value.""" + if value is None: + return None + token = value.strip() + if not token or token == "*" * 8: + return None + if any(ord(character) < 32 or ord(character) == 127 for character in token): + raise HTTPException(status_code=422, detail="Noema gateway token is invalid") + return token + + +def _response(config: TenantConfig | None) -> NoemaGatewayResponse: + """Build a response containing only non-secret gateway state.""" + if config is None: + return NoemaGatewayResponse() + has_token = bool(config.noema_orchestrator_token) + return NoemaGatewayResponse( + base_url=config.noema_orchestrator_base_url, + configured=bool(config.noema_orchestrator_base_url and has_token), + has_token=has_token, + ) + + +def _is_encryption_configuration_error(error: BaseException) -> bool: + """Recognize direct or SQLAlchemy-wrapped encryption configuration errors.""" + if isinstance(error, EncryptionConfigurationError): + return True + return isinstance(error, StatementError) and isinstance( + error.orig, EncryptionConfigurationError + ) + + +@router.get("", response_model=NoemaGatewayResponse) +async def get_noema_gateway( + db: AsyncSession = Depends(get_db), + auth_context: AuthContext = Depends(get_auth_context), +) -> NoemaGatewayResponse: + """Return the signed-session user's scoped gateway readiness state.""" + try: + config = await get_scoped_tenant_config( + db, auth_context.user_id, auth_context.organization_id + ) + return _response(config) + except Exception as exc: + if not _is_encryption_configuration_error(exc): + raise + raise HTTPException( + status_code=503, + detail="Server encryption key is not configured. Contact your workspace administrator.", + ) from exc + + +@router.put("", response_model=NoemaGatewayResponse) +async def update_noema_gateway( + update: NoemaGatewayUpdate, + db: AsyncSession = Depends(get_db), + auth_context: AuthContext = Depends(get_auth_context), +) -> NoemaGatewayResponse: + """Persist the current user's gateway settings with an auditable change.""" + updates = update.model_dump(exclude_unset=True) + if not updates: + raise HTTPException(status_code=422, detail="No gateway settings supplied") + + try: + config = await get_scoped_tenant_config( + db, auth_context.user_id, auth_context.organization_id + ) + if config is None: + config = new_scoped_tenant_config( + user_id=auth_context.user_id, + organization_id=auth_context.organization_id, + ) + db.add(config) + + if "token" in updates: + token = _clean_token(updates["token"]) + if token is not None: + config.noema_orchestrator_token = token + elif not config.noema_orchestrator_token: + raise HTTPException( + status_code=422, detail="Noema gateway token is required" + ) + + if "base_url" in updates: + config.noema_orchestrator_base_url = await _validated_base_url( + updates["base_url"] or "" + ) + + if not config.noema_orchestrator_base_url or not config.noema_orchestrator_token: + raise HTTPException( + status_code=422, + detail="Noema gateway base URL and token are required", + ) + + resource_uid = _resource_uid(auth_context) + db.add( + AuditLog( + user_id=auth_context.user_id, + action="update", + resource_type="noema_gateway", + resource_id=resource_uid, + details="Updated Noema gateway settings", + ) + ) + db.add( + SecurityAuditEvent( + actor_user_id=auth_context.user_id, + actor_role=auth_context.role, + organization_id=auth_context.organization_id, + workspace_id=auth_context.workspace_id, + event_action="update", + resource_type="noema_gateway", + resource_uid=resource_uid, + evidence_source="api.noema_config", + detail_text="Updated Noema gateway settings", + ) + ) + await db.commit() + return _response(config) + except Exception as exc: + if not _is_encryption_configuration_error(exc): + raise + raise HTTPException( + status_code=503, + detail="Server encryption key is not configured. Contact your workspace administrator.", + ) from exc diff --git a/backend/core/runtime_secrets.py b/backend/core/runtime_secrets.py index 1bf1496d1..d484bcc8f 100644 --- a/backend/core/runtime_secrets.py +++ b/backend/core/runtime_secrets.py @@ -18,6 +18,10 @@ ENCRYPTION_KEY_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.-]{0,63}$") +class EncryptionConfigurationError(RuntimeError): + """Raised when the runtime cannot encrypt or decrypt protected settings.""" + + @dataclass(frozen=True) class RuntimeEncryptionKey: key_id: str @@ -147,7 +151,7 @@ def build_encryption_keyring( previous_keys_value: str | None = None, ) -> EncryptionKeyRing: if active_key_value is None or not active_key_value.strip(): - raise RuntimeError( + raise EncryptionConfigurationError( "ENCRYPTION_KEY is required. Refusing to encrypt without a configured key." ) diff --git a/backend/db/models.py b/backend/db/models.py index 98e17eef2..532dd3955 100644 --- a/backend/db/models.py +++ b/backend/db/models.py @@ -1362,6 +1362,16 @@ class TenantConfig(Base): batch_attribution_group: Mapped[str | None] = mapped_column(String, nullable=True) batch_attribution_company: Mapped[str | None] = mapped_column(String, nullable=True) + # Dedicated contextual-orchestrator inference gateway for in-process Noema + # judgments. Token is EncryptedString (Fernet KV); URL is SSRF-guarded at + # call time. Upstream provider keys stay in the orchestrator KV. + noema_orchestrator_base_url: Mapped[str | None] = mapped_column( + String, nullable=True + ) + noema_orchestrator_token: Mapped[str | None] = mapped_column( + EncryptedString, nullable=True + ) + def __repr__(self) -> str: return ( f" list[Executable]: text( "ALTER TABLE tenant_configs ADD COLUMN IF NOT EXISTS pop3_password varchar" ), + text( + "ALTER TABLE tenant_configs " + "ADD COLUMN IF NOT EXISTS noema_orchestrator_base_url varchar" + ), + text( + "ALTER TABLE tenant_configs " + "ADD COLUMN IF NOT EXISTS noema_orchestrator_token varchar" + ), text( "ALTER TABLE sender_relationships " "ADD COLUMN IF NOT EXISTS source_message_id varchar" diff --git a/backend/services/agent_registry.py b/backend/services/agent_registry.py index bf8d94a72..f567bb5c8 100644 --- a/backend/services/agent_registry.py +++ b/backend/services/agent_registry.py @@ -39,6 +39,8 @@ class RegisteredAgent: description: str = "" capabilities: tuple[str, ...] = () provider_source: str = "" + model_alias: str = "" + sequential_failover: bool = False writeback_opt_in: bool = False writeback_audit_logged: bool = False degrades_gracefully: bool = False @@ -75,6 +77,8 @@ def _agent_from_entry(agent_id: str, entry: dict[str, Any]) -> RegisteredAgent | description=str(entry.get("description", "") or ""), capabilities=_coerce_capabilities(entry.get("capabilities")), provider_source=str(entry.get("provider_source", "") or ""), + model_alias=str(entry.get("model_alias", "") or ""), + sequential_failover=bool(entry.get("sequential_failover", False)), writeback_opt_in=bool(writeback.get("opt_in", False)), writeback_audit_logged=bool(writeback.get("audit_logged", False)), degrades_gracefully=bool(entry.get("degrades_gracefully", False)), diff --git a/backend/services/llm_provider_selection.py b/backend/services/llm_provider_selection.py index a10c8e11e..10025d5f5 100644 --- a/backend/services/llm_provider_selection.py +++ b/backend/services/llm_provider_selection.py @@ -90,6 +90,12 @@ async def resolve_runtime_llm_provider( user_id: str, organization_id: str | None, ) -> RuntimeLLMProvider | None: + """Resolve the tenant Fernet LLM provider for search / chat / embeddings. + + Noema is not a caller. Decision-agent completions go through + ``services.orchestrator_gateway.resolve_orchestrator_gateway`` so naruon + never places NVIDIA / OpenAI / OpenRouter / Bytez keys on that path. + """ active_provider = await get_active_llm_provider(session, organization_id) if active_provider is not None: runtime_provider = _runtime_from_provider(active_provider) diff --git a/backend/services/noema_agent.py b/backend/services/noema_agent.py index 99ab683df..805b168d7 100644 --- a/backend/services/noema_agent.py +++ b/backend/services/noema_agent.py @@ -1,10 +1,16 @@ """Noema general agent. -A general-purpose `Pydantic-AI `_ (MIT) agent that -reasons over the naruon workspace. It runs on the tenant's configured LLM -provider — resolved through :func:`resolve_runtime_llm_provider` from the -Fernet-encrypted provider records, never from ``os.getenv`` — and it is given a -small set of tools that plug into the existing service and runner seams: +An in-process `Pydantic-AI `_ (MIT) agent that naruon +keeps as the existing tool surface (mail, content graph, tasks, opt-in writeback). +LLM calls go only to the contextual-orchestrator gateway: dedicated inference +token + HTTPS ``/v1`` base URL from the Fernet tenant KV, and the single model +alias ``contextual-orchestrator``. naruon does not resolve a tenant LLM +provider record, does not pick a tenant chat-model name, does not hold +upstream provider keys at request time, and does not sequentially fail over +to the next model. Catalog mappings stay catalog-only; this module does not +dispatch Decision Points or ``mail.triage``. + +Tools stay owner-scoped: * **read/search mail** and **content-graph queries** are owner-scoped SQL reads. * **task actions** update ``TicketTask`` rows and are audit-logged. @@ -13,9 +19,9 @@ naruon's opt-in-writeback and audit-logged contract. Pydantic-AI is an *optional* runtime dependency (installed via -``backend/requirements-agent.txt``). When it — or a usable LLM provider — is not -available the agent degrades gracefully: it returns a structured no-op notice -instead of raising, so the surrounding request path stays healthy. +``backend/requirements-agent.txt``). When it — or the orchestrator gateway — is +not available the agent degrades gracefully: it returns a structured no-op +notice instead of raising, so the surrounding request path stays healthy. """ from __future__ import annotations @@ -25,6 +31,7 @@ from dataclasses import dataclass, field from typing import TYPE_CHECKING, Any, Awaitable, Callable, Literal +from openai import AsyncOpenAI from sqlalchemy import or_, select from sqlalchemy.ext.asyncio import AsyncSession @@ -35,11 +42,12 @@ KnowledgeGraphEdgeRecord, TicketTask, ) -from services.llm_provider_selection import ( - RuntimeLLMProvider, - resolve_runtime_llm_provider, -) from services.llm_provider_urls import build_llm_provider_http_client +from services.orchestrator_gateway import ( + ORCHESTRATOR_MODEL_ALIAS, + OrchestratorGateway, + resolve_orchestrator_gateway, +) if TYPE_CHECKING: # pragma: no cover - typing only from pydantic_ai import Agent @@ -100,6 +108,8 @@ class NoemaAgentResult: output: str = "" notice: str | None = None provider_name: str | None = None + model_alias: str | None = None + error_code: str | None = None tool_calls: tuple[str, ...] = () @property @@ -434,11 +444,13 @@ async def _default_dispatcher( SYSTEM_PROMPT = ( "You are Noema, the general assistant for a naruon email workspace. " - "Use the provided tools to read and search the owner's mail, inspect the " - "content graph of an email, and manage tasks. Only change task status or " - "dispatch a writeback when the user clearly asks for it. Writebacks target " - "the customer's own systems and require opt-in; if a writeback is skipped, " - "explain that it must be enabled. Be concise and cite message ids you used." + "Do not try alternate models or providers; contextual-orchestrator " + "selects the model. Use the provided tools to read and search the " + "owner's mail, inspect the content graph of an email, and manage tasks. " + "Only change task status or dispatch a writeback when the user clearly " + "asks for it. Writebacks target the customer's own systems and require " + "opt-in; if a writeback is skipped, explain that it must be enabled. " + "Be concise and cite message ids you used." ) @@ -462,15 +474,24 @@ def _load_pydantic_ai() -> Any | None: } +async def _aclose_http_client(http_client: Any) -> None: + closer = getattr(http_client, "aclose", None) + if closer is None: + return + await closer() + + async def build_noema_agent( - provider: RuntimeLLMProvider, + gateway: OrchestratorGateway, ) -> tuple["Agent | None", Callable[[], Awaitable[None]]]: - """Build the pydantic-ai agent for a resolved provider. + """Build the pydantic-ai agent for the orchestrator gateway. Returns ``(agent, closer)``. ``agent`` is ``None`` when pydantic-ai is not - installed; ``closer`` always closes any opened HTTP client. + installed; ``closer`` always closes any opened HTTP client. The model name + is always :data:`ORCHESTRATOR_MODEL_ALIAS` — never a sequential list. + A rejected or missing gateway URL raises ``ValueError`` with + ``orchestrator_gateway_unavailable`` and never constructs ``AsyncOpenAI``. """ - from openai import AsyncOpenAI async def _noop_closer() -> None: return None @@ -479,11 +500,19 @@ async def _noop_closer() -> None: if modules is None: return None, _noop_closer - validated_base_url, http_client = await build_llm_provider_http_client( - provider.base_url - ) + try: + validated_base_url, http_client = await build_llm_provider_http_client( + gateway.base_url + ) + except ValueError as exc: + raise ValueError("orchestrator_gateway_unavailable") from exc + + if not validated_base_url: + await _aclose_http_client(http_client) + raise ValueError("orchestrator_gateway_unavailable") + openai_client = AsyncOpenAI( - api_key=provider.api_key, + api_key=gateway.inference_token, base_url=validated_base_url, http_client=http_client, ) @@ -492,7 +521,7 @@ async def _closer() -> None: await openai_client.close() model = modules["OpenAIChatModel"]( - provider.chat_model, + gateway.model_alias, provider=modules["OpenAIProvider"](openai_client=openai_client), ) agent = modules["Agent"]( @@ -560,25 +589,37 @@ async def run_noema_agent( writeback_enabled: bool = False, dispatcher: RunnerDispatcher | None = None, ) -> NoemaAgentResult: - """Run the Noema general agent, degrading gracefully when unavailable. + """Run the Noema decision agent, degrading gracefully when unavailable. This is the entrypoint referenced by ``registered_agents.json``. """ - provider = await resolve_runtime_llm_provider( + gateway = await resolve_orchestrator_gateway( session, user_id=user_id, organization_id=organization_id ) - if provider is None: + if gateway is None: return NoemaAgentResult( status="unavailable", - notice="No LLM provider is configured for this workspace.", + notice="The contextual-orchestrator gateway is not configured.", + error_code="orchestrator_gateway_unavailable", ) - agent, closer = await build_noema_agent(provider) + try: + agent, closer = await build_noema_agent(gateway) + except ValueError: + return NoemaAgentResult( + status="unavailable", + notice="The contextual-orchestrator gateway is not configured.", + provider_name=ORCHESTRATOR_MODEL_ALIAS, + model_alias=gateway.model_alias, + error_code="orchestrator_gateway_unavailable", + ) if agent is None: return NoemaAgentResult( status="unavailable", notice="The pydantic-ai runtime is not installed; agent is disabled.", - provider_name=provider.provider_name, + provider_name=ORCHESTRATOR_MODEL_ALIAS, + model_alias=gateway.model_alias, + error_code="noema_runtime_unavailable", ) deps = NoemaAgentDeps( @@ -594,7 +635,8 @@ async def run_noema_agent( return NoemaAgentResult( status="ok", output=str(getattr(result, "output", "")), - provider_name=provider.provider_name, + provider_name=ORCHESTRATOR_MODEL_ALIAS, + model_alias=gateway.model_alias, tool_calls=tuple(deps.tool_calls), ) except Exception as exc: # noqa: BLE001 - degrade gracefully, never propagate @@ -602,7 +644,9 @@ async def run_noema_agent( return NoemaAgentResult( status="error", notice="The agent run could not be completed.", - provider_name=provider.provider_name, + provider_name=ORCHESTRATOR_MODEL_ALIAS, + model_alias=gateway.model_alias, + error_code="noema_run_failed", tool_calls=tuple(deps.tool_calls), ) finally: diff --git a/backend/services/orchestrator_gateway.py b/backend/services/orchestrator_gateway.py new file mode 100644 index 000000000..b0bd298a1 --- /dev/null +++ b/backend/services/orchestrator_gateway.py @@ -0,0 +1,139 @@ +"""contextual-orchestrator inference gateway for in-process agents. + +naruon is a consumer of ContextualWisdomLab/contextual-orchestrator. This +module resolves the dedicated gateway inference token and the HTTPS ``/v1`` +base URL from the Fernet-encrypted tenant credential store (the KV). It does +not read ``os.getenv`` at request time, does not hold upstream provider keys, +and does not pick or fail over across models. The orchestrator auto-discovers +upstream models and itself selects min-cost / max-performance (Fugu / +Conductor / TRINITY); naruon always sends the single model alias +``contextual-orchestrator``. + +Upstream org secrets (``NVIDIA_NIM_API_KEY``, ``NVIDIA_NIM_API_KEY_SUB``, +``BYTEZ_API_KEY``, ``OPENROUTER_API_KEY``, ``OPENAI_API_KEY``) belong in the +orchestrator KV, not in naruon request-time config. GitHub Models / +``COPILOT_GITHUB_TOKEN`` are never a Noema path. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from urllib.parse import urlsplit, urlunsplit + +from sqlalchemy.ext.asyncio import AsyncSession + +from services.llm_provider_urls import validate_llm_provider_base_url_async +from services.tenant_config_scope import get_scoped_tenant_config + +ORCHESTRATOR_MODEL_ALIAS = "contextual-orchestrator" + +# Named so tests can prove naruon never consumes these at request time. +# They are registered in the orchestrator KV, not in naruon. +UPSTREAM_PROVIDER_SECRET_NAMES = frozenset( + { + "NVIDIA_NIM_API_KEY", + "NVIDIA_NIM_API_KEY_SUB", + "BYTEZ_API_KEY", + "OPENROUTER_API_KEY", + "OPENAI_API_KEY", + "COPILOT_GITHUB_TOKEN", + } +) + +FORBIDDEN_GATEWAY_HOSTS = frozenset( + { + "models.github.ai", + "api.githubcopilot.com", + "copilot-proxy.githubusercontent.com", + } +) + + +@dataclass(frozen=True) +class OrchestratorGateway: + """Single-alias OpenAI-compatible gateway naruon may call.""" + + inference_token: str + base_url: str + model_alias: str = ORCHESTRATOR_MODEL_ALIAS + model_candidates: tuple[str, ...] = () + + +def _clean(value: str | None) -> str | None: + if value is None: + return None + stripped = value.strip() + return stripped or None + + +def validate_orchestrator_gateway_url(value: str) -> str: + """Require HTTPS and a path that ends in ``/v1``; reject GitHub Models.""" + candidate = (value or "").strip() + if not candidate: + raise ValueError("orchestrator gateway URL is required") + + parsed = urlsplit(candidate) + hostname = (parsed.hostname or "").lower().rstrip(".") + if parsed.scheme.lower() != "https": + raise ValueError("orchestrator gateway URL must be HTTPS") + if ( + not hostname + or parsed.username is not None + or parsed.password is not None + or parsed.query + or parsed.fragment + ): + raise ValueError("orchestrator gateway URL is not allowed") + if hostname in FORBIDDEN_GATEWAY_HOSTS: + raise ValueError("orchestrator gateway URL is not allowed") + + path = (parsed.path or "").rstrip("/") + if path != "/v1" and not path.endswith("/v1"): + raise ValueError("orchestrator gateway URL must end in /v1") + + return urlunsplit(("https", parsed.netloc.lower(), path, "", "")) + + +async def resolve_orchestrator_gateway( + session: AsyncSession, + *, + user_id: str, + organization_id: str | None, +) -> OrchestratorGateway | None: + """Resolve the dedicated Noema gateway from the Fernet tenant KV. + + Returns ``None`` (fail closed) when the token or HTTPS ``/v1`` URL is + missing or rejected. Never reads the process environment. + """ + tenant_config = await get_scoped_tenant_config(session, user_id, organization_id) + if tenant_config is None: + return None + + base_url = _clean(getattr(tenant_config, "noema_orchestrator_base_url", None)) + token = _clean(getattr(tenant_config, "noema_orchestrator_token", None)) + if not base_url or not token: + return None + + try: + shaped = validate_orchestrator_gateway_url(base_url) + except ValueError: + return None + + try: + allowlisted = await validate_llm_provider_base_url_async(shaped) + except ValueError: + return None + if not allowlisted: + return None + + try: + validated = validate_orchestrator_gateway_url(allowlisted) + except ValueError: + return None + + return OrchestratorGateway( + inference_token=token, + base_url=validated, + model_alias=ORCHESTRATOR_MODEL_ALIAS, + model_candidates=(), + ) diff --git a/backend/tests/test_agent_registry.py b/backend/tests/test_agent_registry.py index 0f451444d..62b512068 100644 --- a/backend/tests/test_agent_registry.py +++ b/backend/tests/test_agent_registry.py @@ -17,7 +17,7 @@ def teardown_function() -> None: clear_registry_cache() -def test_noema_agent_is_registered(): +def test_noema_agent_is_registered_on_orchestrator(): agents = load_registered_agents() assert "noema-general-agent" in agents @@ -27,20 +27,35 @@ def test_noema_agent_is_registered(): assert agent.entrypoint == "services.noema_agent:run_noema_agent" assert agent.enabled is True assert agent.degrades_gracefully is True + assert agent.provider_source == "contextual-orchestrator" + assert agent.model_alias == "contextual-orchestrator" + assert agent.sequential_failover is False + assert agent.provider_source != "runtime_llm_provider" # The opt-in + audit-logged writeback contract is declared in the catalog. assert agent.writeback_opt_in is True assert agent.writeback_audit_logged is True assert "mail.search" in agent.capabilities assert "calendar.writeback" in agent.capabilities + assert "judgment.decide" not in agent.capabilities -def test_task_mapping_resolves_to_noema_agent(): +def test_task_mapping_stays_catalog_only(): mapping = load_task_agent_mapping() assert mapping.get("general") == "noema-general-agent" - - agent = resolve_agent_for_task("mail.triage") - assert agent is not None - assert agent.agent_id == "noema-general-agent" + assert "judgment.decide" not in mapping + + for task_type in ( + "mail.triage", + "mail.search", + "tasks.followup", + "calendar.writeback", + ): + agent = resolve_agent_for_task(task_type) + assert agent is not None + assert agent.agent_id == "noema-general-agent" + assert agent.provider_source == "contextual-orchestrator" + assert agent.model_alias == "contextual-orchestrator" + assert agent.sequential_failover is False def test_unknown_task_type_resolves_to_none(): diff --git a/backend/tests/test_alembic_migrations.py b/backend/tests/test_alembic_migrations.py index f8f3ffeae..8ccfcc90d 100644 --- a/backend/tests/test_alembic_migrations.py +++ b/backend/tests/test_alembic_migrations.py @@ -452,3 +452,19 @@ def test_merge_revision_reconciles_newsdom_document_and_carddav_heads(): assert "op.create_table(" not in revision_text assert "op.add_column(" not in revision_text assert "op.drop_column(" not in revision_text + + +def test_noema_orchestrator_gateway_has_incremental_revision(): + versions_dir = BACKEND_ROOT / "alembic" / "versions" + revision_path = versions_dir / "0018_noema_orch_gateway.py" + assert revision_path.exists() + revision_text = revision_path.read_text() + + assert 'revision = "0018_noema_orch_gateway"' in revision_text + assert 'down_revision = "0017_merge_newsdom_carddav_heads"' in revision_text + assert '"noema_orchestrator_base_url"' in revision_text + assert '"noema_orchestrator_token"' in revision_text + assert "op.add_column(" in revision_text + assert "op.drop_column(" in revision_text + assert "sa.text(" not in revision_text + assert "has_table" in revision_text diff --git a/backend/tests/test_bootstrap_db.py b/backend/tests/test_bootstrap_db.py index 5af0540f0..ad1ae410d 100644 --- a/backend/tests/test_bootstrap_db.py +++ b/backend/tests/test_bootstrap_db.py @@ -422,6 +422,16 @@ def test_schema_backfill_adds_tenant_config_columns_and_indexes(monkeypatch): "alter table tenant_configs add column if not exists pop3_password" in statement for statement in statements ) + assert any( + "alter table tenant_configs add column if not exists " + "noema_orchestrator_base_url" in statement + for statement in statements + ) + assert any( + "alter table tenant_configs add column if not exists " + "noema_orchestrator_token" in statement + for statement in statements + ) assert any( "alter table tenant_configs add column if not exists organization_id" in statement diff --git a/backend/tests/test_noema_agent.py b/backend/tests/test_noema_agent.py index 31a562db6..3685f48d8 100644 --- a/backend/tests/test_noema_agent.py +++ b/backend/tests/test_noema_agent.py @@ -1,13 +1,15 @@ """Fast, mocked tests for the Noema general agent. -These cover three seams without needing a live LLM or a database: +These cover the seams without needing a live LLM or a database: * tool wiring (mail read/search, content-graph, task actions, writeback) -* configuration resolved from the DB provider layer (not ``os.getenv``) -* graceful degradation when the provider or pydantic-ai runtime is absent +* LLM calls go only to the contextual-orchestrator gateway (KV token, not env) +* no sequential model list / provider-key failover +* graceful degradation when the gateway or pydantic-ai runtime is absent """ import datetime +from pathlib import Path import pytest @@ -16,11 +18,9 @@ ContentNodeRecord, Email, KnowledgeGraphEdgeRecord, - LLMProvider, TicketTask, ) from services import noema_agent -from services.llm_provider_selection import RuntimeLLMProvider from services.noema_agent import ( NOEMA_TOOL_SPECS, NoemaAgentDeps, @@ -33,6 +33,10 @@ tool_search_mail, tool_update_task_status, ) +from services.orchestrator_gateway import ( + ORCHESTRATOR_MODEL_ALIAS, + OrchestratorGateway, +) UTC = datetime.timezone.utc @@ -304,48 +308,46 @@ def test_tool_specs_cover_declared_capabilities(): # --------------------------------------------------------------------------- # -# Config-from-DB + graceful degradation +# Orchestrator gateway + graceful degradation # --------------------------------------------------------------------------- # -class _ProviderScalars: - def __init__(self, items): - self._items = items - - def first(self): - return self._items[0] if self._items else None - - -class _ProviderResult: - def __init__(self, providers): - self._providers = providers +def _gateway() -> OrchestratorGateway: + return OrchestratorGateway( + inference_token="naruon-orch-inference-token", + base_url="https://orchestrator.example/v1", + ) - def scalars(self): - return _ProviderScalars(self._providers) - def scalar_one_or_none(self): +@pytest.mark.asyncio +async def test_run_agent_unavailable_without_orchestrator_gateway(monkeypatch): + async def _no_gateway(*args, **kwargs): return None - -class _ProviderSession: - """Minimal session that satisfies resolve_runtime_llm_provider.""" - - def __init__(self, providers): - self._providers = providers - - async def execute(self, statement): - text = str(statement).lower() - if "llm_providers" in text: - return _ProviderResult(self._providers) - return _ProviderResult([]) + monkeypatch.setattr(noema_agent, "resolve_orchestrator_gateway", _no_gateway) + result = await run_noema_agent( + _QueueSession([]), + user_id="user-1", + organization_id="org-1", + workspace_id="workspace-org-1", + prompt="hello", + ) + assert result.status == "unavailable" + assert result.error_code == "orchestrator_gateway_unavailable" + assert result.provider_name is None + assert result.model_alias is None @pytest.mark.asyncio -async def test_run_agent_unavailable_without_provider(monkeypatch): - async def _no_provider(*args, **kwargs): - return None +async def test_run_agent_uses_orchestrator_gateway_and_degrades_without_runtime( + monkeypatch, +): + async def _gateway_from_kv(*args, **kwargs): + return _gateway() + + monkeypatch.setattr(noema_agent, "resolve_orchestrator_gateway", _gateway_from_kv) + monkeypatch.setattr(noema_agent, "_load_pydantic_ai", lambda: None) - monkeypatch.setattr(noema_agent, "resolve_runtime_llm_provider", _no_provider) result = await run_noema_agent( _QueueSession([]), user_id="user-1", @@ -354,53 +356,157 @@ async def _no_provider(*args, **kwargs): prompt="hello", ) assert result.status == "unavailable" - assert result.provider_name is None + assert result.provider_name == ORCHESTRATOR_MODEL_ALIAS + assert result.model_alias == ORCHESTRATOR_MODEL_ALIAS + assert "pydantic-ai" in (result.notice or "") @pytest.mark.asyncio -async def test_run_agent_uses_db_provider_and_degrades_without_runtime(monkeypatch): - provider = LLMProvider( - id=7, +async def test_run_agent_uses_single_orchestrator_alias(monkeypatch): + captured: dict[str, object] = {} + + async def _gateway_from_kv(*args, **kwargs): + return _gateway() + + async def _fake_build(gateway): + captured["gateway"] = gateway + + class _Agent: + async def run(self, prompt, deps): + captured["prompt"] = prompt + return type("Result", (), {"output": "Hold the reply until Friday."})() + + async def _closer() -> None: + return None + + return _Agent(), _closer + + monkeypatch.setattr(noema_agent, "resolve_orchestrator_gateway", _gateway_from_kv) + monkeypatch.setattr(noema_agent, "build_noema_agent", _fake_build) + + result = await run_noema_agent( + _QueueSession([]), user_id="user-1", organization_id="org-1", - name="Local Gemma", - provider_type="ollama", - base_url="http://ollama:11434/v1", - model_identifier="gemma", - embedding_model="embeddinggemma", - api_key=None, - is_active=True, - updated_at=datetime.datetime.now(UTC), + workspace_id="workspace-org-1", + prompt="Should I reply today?", + ) + assert result.status == "ok" + assert result.output == "Hold the reply until Friday." + assert result.model_alias == "contextual-orchestrator" + assert result.error_code is None + gateway = captured["gateway"] + assert isinstance(gateway, OrchestratorGateway) + assert gateway.model_alias == "contextual-orchestrator" + assert gateway.model_candidates == () + assert gateway.inference_token == "naruon-orch-inference-token" + assert gateway.base_url == "https://orchestrator.example/v1" + + +@pytest.mark.asyncio +async def test_build_agent_targets_orchestrator_alias_only(monkeypatch): + captured: dict[str, object] = {} + + class _FakeOpenAI: + def __init__(self, api_key, base_url, http_client=None): + captured["api_key"] = api_key + captured["base_url"] = base_url + + async def close(self): + return None + + class _FakeChatModel: + def __init__(self, model_name, provider=None): + captured["model_name"] = model_name + captured["provider"] = provider + + class _FakeProvider: + def __init__(self, openai_client=None): + captured["openai_client"] = openai_client + + class _FakeAgent: + def __init__(self, model, deps_type=None, system_prompt=None): + captured["system_prompt"] = system_prompt + self.tools = [] + + def tool(self, fn): + self.tools.append(fn.__name__) + return fn + + async def _fake_http_client(base_url): + captured["validated_base_url"] = base_url + return base_url, object() + + monkeypatch.setattr(noema_agent, "_load_pydantic_ai", lambda: { + "Agent": _FakeAgent, + "RunContext": object, + "OpenAIChatModel": _FakeChatModel, + "OpenAIProvider": _FakeProvider, + }) + monkeypatch.setattr(noema_agent, "AsyncOpenAI", _FakeOpenAI) + monkeypatch.setattr(noema_agent, "build_llm_provider_http_client", _fake_http_client) + + agent, closer = await build_noema_agent(_gateway()) + assert agent is not None + assert captured["api_key"] == "naruon-orch-inference-token" + assert captured["base_url"] == "https://orchestrator.example/v1" + assert captured["model_name"] == "contextual-orchestrator" + assert captured["model_name"] != "gpt-4o" + assert "chat_model" not in captured + await closer() + + +@pytest.mark.asyncio +@pytest.mark.parametrize("failure_mode", ["rejected", "missing"]) +async def test_run_agent_fails_closed_for_invalid_gateway_client( + monkeypatch, failure_mode +): + class _FakeClient: + def __init__(self): + self.closed = False + + async def aclose(self): + self.closed = True + + client = _FakeClient() + + async def _gateway_from_kv(*args, **kwargs): + return _gateway() + + async def _invalid_http_client(base_url): + if failure_mode == "rejected": + raise ValueError("rejected") + return None, client + + monkeypatch.setattr(noema_agent, "resolve_orchestrator_gateway", _gateway_from_kv) + monkeypatch.setattr(noema_agent, "_load_pydantic_ai", lambda: {}) + monkeypatch.setattr( + noema_agent, "build_llm_provider_http_client", _invalid_http_client + ) + monkeypatch.setattr( + noema_agent, + "AsyncOpenAI", + lambda **kwargs: pytest.fail("rejected gateway must not build AsyncOpenAI"), ) - # Simulate the pydantic-ai runtime being absent: the config still resolves - # from the DB provider, and the agent degrades to a notice. - monkeypatch.setattr(noema_agent, "_load_pydantic_ai", lambda: None) result = await run_noema_agent( - _ProviderSession([provider]), + _QueueSession([]), user_id="user-1", organization_id="org-1", workspace_id="workspace-org-1", prompt="hello", ) + assert result.status == "unavailable" - # Proves the provider name came from the DB record, not os.getenv. - assert result.provider_name is not None - assert "pydantic-ai" in (result.notice or "") + assert result.error_code == "orchestrator_gateway_unavailable" + assert result.model_alias == ORCHESTRATOR_MODEL_ALIAS + assert client.closed is (failure_mode == "missing") @pytest.mark.asyncio async def test_build_agent_returns_none_without_runtime(monkeypatch): monkeypatch.setattr(noema_agent, "_load_pydantic_ai", lambda: None) - provider = RuntimeLLMProvider( - api_key="sk-test", - base_url=None, - chat_model="gpt-4o", - embedding_model="text-embedding-3-small", - provider_name="OpenAI", - provider_source="tenant_config", - ) - agent, closer = await build_noema_agent(provider) + agent, closer = await build_noema_agent(_gateway()) assert agent is None await closer() # no-op closer must be awaitable @@ -411,7 +517,7 @@ async def test_build_agent_returns_none_without_runtime(monkeypatch): @pytest.mark.asyncio -async def test_agent_runs_tools_with_test_model(): +async def test_agent_runs_tools_with_test_model(monkeypatch): # This is the ONLY test that exercises the real pydantic-ai build path # (imports OpenAIChatModel, constructs the Agent, registers the tools and # their RunContext-typed schemas). It is skipped only when pydantic-ai is @@ -421,15 +527,11 @@ async def test_agent_runs_tools_with_test_model(): from pydantic_ai import Agent as PydanticAgent from pydantic_ai.models.test import TestModel - provider = RuntimeLLMProvider( - api_key="sk-test", - base_url=None, - chat_model="gpt-4o", - embedding_model="text-embedding-3-small", - provider_name="OpenAI", - provider_source="tenant_config", - ) - agent, closer = await build_noema_agent(provider) + async def _fake_http_client(base_url): + return base_url, None + + monkeypatch.setattr(noema_agent, "build_llm_provider_http_client", _fake_http_client) + agent, closer = await build_noema_agent(_gateway()) # A real Agent must be built — not the graceful-degradation None. assert agent is not None assert isinstance(agent, PydanticAgent) @@ -448,3 +550,23 @@ async def test_agent_runs_tools_with_test_model(): expected_tools = {spec["name"] for spec in NOEMA_TOOL_SPECS} assert expected_tools <= set(deps.tool_calls) assert getattr(result, "output", None) is not None + + +def test_noema_agent_source_does_not_import_tenant_llm_provider(): + """Unique slice: Noema's LLM client is orchestrator-only. + + ``resolve_runtime_llm_provider`` remains for search / chat / embeddings. + This file must not import that resolver or ``llm_provider_selection``. + """ + source = Path(noema_agent.__file__).read_text(encoding="utf-8") + assert "llm_provider_selection" not in source + assert "resolve_runtime_llm_provider" not in source + assert "RuntimeLLMProvider" not in source + assert "model_profile_id" not in source + assert "contextual_orchestrator_client" not in source + assert "gpt-4o" not in source + assert "from services.orchestrator_gateway import" in source + assert "run_noema_decision" not in source + assert "COPILOT_GITHUB_TOKEN" not in source + assert "models.github.ai" not in source + assert "api.githubcopilot.com" not in source diff --git a/backend/tests/test_noema_agent_gateway_fail_closed.py b/backend/tests/test_noema_agent_gateway_fail_closed.py new file mode 100644 index 000000000..50e161554 --- /dev/null +++ b/backend/tests/test_noema_agent_gateway_fail_closed.py @@ -0,0 +1,87 @@ +"""Fail-closed regressions for Noema's contextual-orchestrator client path.""" + +from __future__ import annotations + +from typing import Any + +import pytest + +from services import noema_agent +from services.noema_agent import build_noema_agent, run_noema_agent +from services.orchestrator_gateway import OrchestratorGateway + + +def _gateway() -> OrchestratorGateway: + """Return one dedicated contextual-orchestrator gateway fixture.""" + return OrchestratorGateway( + inference_token="naruon-orch-inference-token", + base_url="https://orchestrator.example/v1", + ) + + +@pytest.mark.asyncio +async def test_run_agent_maps_http_client_validation_failure_to_gateway_unavailable( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """DNS/allowlist validation failures never escape the public entrypoint.""" + + async def _gateway_from_kv(*_args: Any, **_kwargs: Any) -> OrchestratorGateway: + return _gateway() + + async def _validation_failure(_base_url: str) -> tuple[str, object]: + raise ValueError("host_not_allowlisted: orchestrator.example") + + monkeypatch.setattr(noema_agent, "resolve_orchestrator_gateway", _gateway_from_kv) + monkeypatch.setattr(noema_agent, "_load_pydantic_ai", lambda: {"runtime": object()}) + monkeypatch.setattr( + noema_agent, + "build_llm_provider_http_client", + _validation_failure, + ) + + result = await run_noema_agent( + object(), # type: ignore[arg-type] + user_id="user-1", + organization_id="org-1", + workspace_id="workspace-org-1", + prompt="hello", + ) + + assert result.status == "unavailable" + assert result.error_code == "orchestrator_gateway_unavailable" + assert result.provider_name == "contextual-orchestrator" + assert result.model_alias == "contextual-orchestrator" + assert "host_not_allowlisted" not in (result.notice or "") + + +@pytest.mark.asyncio +async def test_build_agent_rejects_missing_base_url_before_openai_client_creation( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A rejected URL closes its pinned client and cannot fall back to OpenAI.""" + + state = {"closed": False, "openai_created": False} + + class _PinnedClient: + async def aclose(self) -> None: + state["closed"] = True + + async def _missing_base_url(_base_url: str) -> tuple[None, _PinnedClient]: + return None, _PinnedClient() + + def _unexpected_openai(*_args: Any, **_kwargs: Any) -> object: + state["openai_created"] = True + return object() + + monkeypatch.setattr(noema_agent, "_load_pydantic_ai", lambda: {"runtime": object()}) + monkeypatch.setattr( + noema_agent, + "build_llm_provider_http_client", + _missing_base_url, + ) + monkeypatch.setattr(noema_agent, "AsyncOpenAI", _unexpected_openai) + + with pytest.raises(ValueError, match="orchestrator_gateway_unavailable"): + await build_noema_agent(_gateway()) + + assert state == {"closed": True, "openai_created": False} diff --git a/backend/tests/test_noema_config_api.py b/backend/tests/test_noema_config_api.py new file mode 100644 index 000000000..ba2a21669 --- /dev/null +++ b/backend/tests/test_noema_config_api.py @@ -0,0 +1,396 @@ +import pytest +from httpx import ASGITransport, AsyncClient +from sqlalchemy.exc import StatementError + +from api.auth import AuthContext, get_auth_context +from core.runtime_secrets import EncryptionConfigurationError +from db.models import AuditLog, SecurityAuditEvent, TenantConfig +from db.session import get_db +from main import app + + +class _Result: + def __init__(self, value): + self.value = value + + def scalar_one_or_none(self): + return self.value + + +class _Session: + def __init__(self, config=None, commit_error=None): + self.config = config + self.commit_error = commit_error + self.added = [] + self.committed = False + + async def execute(self, _query): + return _Result(self.config) + + def add(self, value): + self.added.append(value) + if isinstance(value, TenantConfig): + self.config = value + + async def commit(self): + self.committed = True + if self.commit_error is not None: + raise self.commit_error + + +@pytest.fixture +def auth_context(): + return AuthContext( + user_id="user-1", + role="member", + organization_id="org-1", + group_ids=(), + workspace_id="workspace-org-1", + ) + + +@pytest.fixture +def override_dependencies(auth_context): + session = _Session() + + async def get_test_db(): + yield session + + async def get_test_auth(): + return auth_context + + app.dependency_overrides[get_db] = get_test_db + app.dependency_overrides[get_auth_context] = get_test_auth + yield session + app.dependency_overrides.pop(get_db, None) + app.dependency_overrides.pop(get_auth_context, None) + + +async def _client(): + return AsyncClient(transport=ASGITransport(app=app), base_url="http://test") + + +async def _allow_gateway_url(value): + return value + + +@pytest.mark.asyncio +async def test_noema_gateway_update_masks_token_and_writes_audit( + override_dependencies, monkeypatch +): + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", + _allow_gateway_url, + ) + + async with await _client() as client: + response = await client.put( + "/api/noema-gateway", + json={ + "base_url": "https://orchestrator.example/v1", + "token": "gateway-secret", + }, + ) + + assert response.status_code == 200 + assert response.json() == { + "base_url": "https://orchestrator.example/v1", + "configured": True, + "has_token": True, + } + config = override_dependencies.config + assert config.noema_orchestrator_token == "gateway-secret" + assert "gateway-secret" not in response.text + assert any(isinstance(item, AuditLog) for item in override_dependencies.added) + event = next( + item + for item in override_dependencies.added + if isinstance(item, SecurityAuditEvent) + ) + assert event.resource_type == "noema_gateway" + assert event.detail_text == "Updated Noema gateway settings" + assert "gateway-secret" not in event.detail_text + + +@pytest.mark.asyncio +async def test_noema_gateway_get_returns_readiness_without_secret( + override_dependencies, monkeypatch +): + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", + _allow_gateway_url, + ) + override_dependencies.config = TenantConfig( + user_id="user-1", + organization_id="org-1", + noema_orchestrator_base_url="https://orchestrator.example/v1", + noema_orchestrator_token="gateway-secret", + ) + + async with await _client() as client: + response = await client.get("/api/noema-gateway") + + assert response.status_code == 200 + assert response.json() == { + "base_url": "https://orchestrator.example/v1", + "configured": True, + "has_token": True, + } + assert "gateway-secret" not in response.text + + +@pytest.mark.asyncio +async def test_noema_gateway_get_without_config_is_not_ready(override_dependencies): + async with await _client() as client: + response = await client.get("/api/noema-gateway") + + assert response.status_code == 200 + assert response.json() == { + "base_url": None, + "configured": False, + "has_token": False, + } + + +@pytest.mark.asyncio +async def test_noema_gateway_get_handles_encryption_configuration_failure( + override_dependencies, monkeypatch +): + async def fail_to_read_config(*_args): + raise EncryptionConfigurationError("ENCRYPTION_KEY is required") + + monkeypatch.setattr( + "api.noema_config.get_scoped_tenant_config", fail_to_read_config + ) + async with await _client() as client: + response = await client.get("/api/noema-gateway") + + assert response.status_code == 503 + assert "Server encryption key is not configured" in response.json()["detail"] + + +@pytest.mark.asyncio +async def test_noema_gateway_update_handles_encryption_configuration_failure_while_reading( + override_dependencies, monkeypatch +): + async def fail_to_read_config(*_args): + raise EncryptionConfigurationError("ENCRYPTION_KEY is required") + + monkeypatch.setattr( + "api.noema_config.get_scoped_tenant_config", fail_to_read_config + ) + async with await _client() as client: + response = await client.put( + "/api/noema-gateway", + json={ + "base_url": "https://orchestrator.example/v1", + "token": "gateway-secret", + }, + ) + + assert response.status_code == 503 + assert "Server encryption key is not configured" in response.json()["detail"] + assert not override_dependencies.committed + + +@pytest.mark.asyncio +async def test_noema_gateway_rejects_unallowlisted_normalized_url( + override_dependencies, monkeypatch +): + async def reject_url(_value): + return None + + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", reject_url + ) + async with await _client() as client: + response = await client.put( + "/api/noema-gateway", + json={ + "base_url": "https://orchestrator.example/v1", + "token": "gateway-secret", + }, + ) + + assert response.status_code == 422 + assert response.json()["detail"] == "Noema gateway base URL is not allowed" + + +@pytest.mark.asyncio +@pytest.mark.parametrize("token", [None, "********"]) +async def test_noema_gateway_preserves_existing_token_for_masked_or_null_input( + override_dependencies, monkeypatch, token +): + override_dependencies.config = TenantConfig( + user_id="user-1", + organization_id="org-1", + noema_orchestrator_base_url="https://orchestrator.example/v1", + noema_orchestrator_token="gateway-secret", + ) + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", + _allow_gateway_url, + ) + async with await _client() as client: + response = await client.put( + "/api/noema-gateway", + json={"base_url": "https://orchestrator.example/v1", "token": token}, + ) + + assert response.status_code == 200 + assert override_dependencies.config.noema_orchestrator_token == "gateway-secret" + + +@pytest.mark.asyncio +async def test_noema_gateway_rejects_control_character_in_token( + override_dependencies +): + async with await _client() as client: + response = await client.put( + "/api/noema-gateway", + json={"token": "gateway\nsecret"}, + ) + + assert response.status_code == 422 + assert response.json()["detail"] == "Noema gateway token is invalid" + + +@pytest.mark.asyncio +async def test_noema_gateway_reports_missing_token_for_valid_url( + override_dependencies, monkeypatch +): + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", + _allow_gateway_url, + ) + async with await _client() as client: + response = await client.put( + "/api/noema-gateway", + json={"base_url": "https://orchestrator.example/v1"}, + ) + + assert response.status_code == 422 + assert response.json()["detail"] == "Noema gateway base URL and token are required" + + +@pytest.mark.asyncio +async def test_noema_gateway_handles_missing_encryption_key( + override_dependencies, monkeypatch +): + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", + _allow_gateway_url, + ) + override_dependencies.commit_error = EncryptionConfigurationError( + "ENCRYPTION_KEY is required" + ) + async with await _client() as client: + response = await client.put( + "/api/noema-gateway", + json={ + "base_url": "https://orchestrator.example/v1", + "token": "gateway-secret", + }, + ) + + assert response.status_code == 503 + assert "Server encryption key is not configured" in response.json()["detail"] + + +@pytest.mark.asyncio +async def test_noema_gateway_handles_wrapped_encryption_configuration_error( + override_dependencies, monkeypatch +): + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", + _allow_gateway_url, + ) + override_dependencies.commit_error = StatementError( + "commit failed", + None, + None, + EncryptionConfigurationError("ENCRYPTION_KEY is required"), + ) + async with await _client() as client: + response = await client.put( + "/api/noema-gateway", + json={ + "base_url": "https://orchestrator.example/v1", + "token": "gateway-secret", + }, + ) + + assert response.status_code == 503 + + +@pytest.mark.asyncio +async def test_noema_gateway_does_not_match_encryption_error_text( + override_dependencies, monkeypatch +): + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", + _allow_gateway_url, + ) + override_dependencies.commit_error = RuntimeError("ENCRYPTION_KEY is required downstream") + with pytest.raises(RuntimeError, match="ENCRYPTION_KEY is required downstream"): + async with await _client() as client: + await client.put( + "/api/noema-gateway", + json={ + "base_url": "https://orchestrator.example/v1", + "token": "gateway-secret", + }, + ) + + +@pytest.mark.asyncio +async def test_noema_gateway_propagates_unexpected_commit_error( + override_dependencies, monkeypatch +): + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", + _allow_gateway_url, + ) + override_dependencies.commit_error = RuntimeError("database unavailable") + with pytest.raises(RuntimeError, match="database unavailable"): + async with await _client() as client: + await client.put( + "/api/noema-gateway", + json={ + "base_url": "https://orchestrator.example/v1", + "token": "gateway-secret", + }, + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + "payload,detail", + [ + ({"base_url": "https://orchestrator.example/v2"}, "Noema gateway base URL is not allowed"), + ({"token": ""}, "Noema gateway token is required"), + ({"unknown": "value"}, "Extra inputs are not permitted"), + ], +) +async def test_noema_gateway_rejects_invalid_updates( + override_dependencies, monkeypatch, payload, detail +): + monkeypatch.setattr( + "api.noema_config.validate_llm_provider_base_url_async", + _allow_gateway_url, + ) + async with await _client() as client: + response = await client.put("/api/noema-gateway", json=payload) + + assert response.status_code == 422 + assert detail in response.text + assert not override_dependencies.committed + + +@pytest.mark.asyncio +async def test_noema_gateway_requires_a_setting_update(override_dependencies): + async with await _client() as client: + response = await client.put("/api/noema-gateway", json={}) + + assert response.status_code == 422 + assert response.json()["detail"] == "No gateway settings supplied" diff --git a/backend/tests/test_orchestrator_gateway.py b/backend/tests/test_orchestrator_gateway.py new file mode 100644 index 000000000..f310cd031 --- /dev/null +++ b/backend/tests/test_orchestrator_gateway.py @@ -0,0 +1,195 @@ +"""Tests for the contextual-orchestrator inference gateway resolver. + +Noema (and any other in-process decision agent) may call only this gateway. +These tests prove the consumer-side contract: + +* dedicated gateway inference token + HTTPS ``/v1`` base URL from the Fernet KV +* model alias is always ``contextual-orchestrator`` (no sequential model list) +* upstream provider keys and GitHub Models tokens are never read at request time +""" + +from __future__ import annotations + +import os +from pathlib import Path + +import pytest +from cryptography.fernet import Fernet +from pydantic import SecretStr +from sqlalchemy import create_engine, text +from sqlalchemy.orm import Session + +from core.config import settings +from db.models import TenantConfig +from services.orchestrator_gateway import ( + FORBIDDEN_GATEWAY_HOSTS, + ORCHESTRATOR_MODEL_ALIAS, + UPSTREAM_PROVIDER_SECRET_NAMES, + OrchestratorGateway, + resolve_orchestrator_gateway, + validate_orchestrator_gateway_url, +) + +ORCH_URL = "https://orchestrator.example/v1" +GATEWAY_TOKEN = "naruon-orch-inference-token" + + +class _FakeResult: + def __init__(self, tenant_config): + self._tenant_config = tenant_config + + def scalar_one_or_none(self): + return self._tenant_config + + +class _FakeAsyncSession: + def __init__(self, tenant_config=None): + self.tenant_config = tenant_config + + async def execute(self, _stmt): + return _FakeResult(self.tenant_config) + + +@pytest.fixture(autouse=True) +def encryption_key(): + old_key = settings.ENCRYPTION_KEY + settings.ENCRYPTION_KEY = SecretStr(Fernet.generate_key().decode("ascii")) + yield + settings.ENCRYPTION_KEY = old_key + + +def test_model_alias_is_the_single_orchestrator_name(): + assert ORCHESTRATOR_MODEL_ALIAS == "contextual-orchestrator" + gateway = OrchestratorGateway( + inference_token=GATEWAY_TOKEN, + base_url=ORCH_URL, + ) + assert gateway.model_alias == "contextual-orchestrator" + assert gateway.model_candidates == () + + +def test_validate_gateway_url_requires_https_v1_suffix(): + assert validate_orchestrator_gateway_url(ORCH_URL) == ORCH_URL + assert validate_orchestrator_gateway_url("https://orchestrator.example/v1/") == ( + "https://orchestrator.example/v1" + ) + with pytest.raises(ValueError): + validate_orchestrator_gateway_url("http://orchestrator.example/v1") + with pytest.raises(ValueError): + validate_orchestrator_gateway_url("https://orchestrator.example/openai") + with pytest.raises(ValueError): + validate_orchestrator_gateway_url("https://orchestrator.example") + + +@pytest.mark.parametrize("host", sorted(FORBIDDEN_GATEWAY_HOSTS)) +def test_validate_gateway_url_rejects_github_models_hosts(host): + with pytest.raises(ValueError): + validate_orchestrator_gateway_url(f"https://{host}/v1") + + +def test_upstream_provider_secret_names_are_denylisted_not_consumed(): + assert "NVIDIA_NIM_API_KEY" in UPSTREAM_PROVIDER_SECRET_NAMES + assert "NVIDIA_NIM_API_KEY_SUB" in UPSTREAM_PROVIDER_SECRET_NAMES + assert "BYTEZ_API_KEY" in UPSTREAM_PROVIDER_SECRET_NAMES + assert "OPENROUTER_API_KEY" in UPSTREAM_PROVIDER_SECRET_NAMES + assert "OPENAI_API_KEY" in UPSTREAM_PROVIDER_SECRET_NAMES + assert "COPILOT_GITHUB_TOKEN" in UPSTREAM_PROVIDER_SECRET_NAMES + + +def test_gateway_modules_do_not_read_env_or_hold_provider_keys(): + roots = [ + Path(__file__).resolve().parents[1] / "services" / "orchestrator_gateway.py", + Path(__file__).resolve().parents[1] / "services" / "noema_agent.py", + ] + for path in roots: + source = path.read_text(encoding="utf-8") + assert "os.getenv(" not in source + assert "os.environ[" not in source + assert "os.environ.get(" not in source + for secret_name in UPSTREAM_PROVIDER_SECRET_NAMES: + # The denylist constant may name the secrets; runtime reads must not. + if path.name == "orchestrator_gateway.py": + continue + assert secret_name not in source + + +@pytest.mark.asyncio +async def test_resolve_gateway_from_fernet_kv_not_env(monkeypatch): + for name in UPSTREAM_PROVIDER_SECRET_NAMES: + monkeypatch.setenv(name, f"env-leak-{name}") + async def allowlisted(value): + return value + + monkeypatch.setattr( + "services.orchestrator_gateway.validate_llm_provider_base_url_async", + allowlisted, + ) + + engine = create_engine("sqlite:///:memory:") + TenantConfig.__table__.create(engine) + try: + with Session(engine) as session: + session.add( + TenantConfig( + user_id="user-1", + organization_id="org-acme", + noema_orchestrator_base_url=ORCH_URL, + noema_orchestrator_token=GATEWAY_TOKEN, + openai_api_key="tenant-openai-must-not-be-used", + ) + ) + session.commit() + raw_token = session.execute( + text("SELECT noema_orchestrator_token FROM tenant_configs") + ).scalar_one() + assert raw_token != GATEWAY_TOKEN + reloaded = session.query(TenantConfig).one() + finally: + engine.dispose() + + resolved = await resolve_orchestrator_gateway( + _FakeAsyncSession(reloaded), + user_id="user-1", + organization_id="org-acme", + ) + assert resolved is not None + assert resolved.base_url == ORCH_URL + assert resolved.inference_token == GATEWAY_TOKEN + assert resolved.model_alias == "contextual-orchestrator" + assert resolved.model_candidates == () + assert resolved.inference_token != os.environ["OPENAI_API_KEY"] + assert resolved.inference_token != os.environ["NVIDIA_NIM_API_KEY"] + assert resolved.inference_token != "tenant-openai-must-not-be-used" + + +@pytest.mark.asyncio +async def test_resolve_gateway_unavailable_when_kv_missing(): + resolved = await resolve_orchestrator_gateway( + _FakeAsyncSession(None), + user_id="user-1", + organization_id="org-acme", + ) + assert resolved is None + + +@pytest.mark.asyncio +async def test_resolve_gateway_rejects_invalid_url(monkeypatch): + async def allowlisted(value): + return value + + monkeypatch.setattr( + "services.orchestrator_gateway.validate_llm_provider_base_url_async", + allowlisted, + ) + config = TenantConfig( + user_id="user-1", + organization_id="org-acme", + noema_orchestrator_base_url="https://models.github.ai/v1", + noema_orchestrator_token=GATEWAY_TOKEN, + ) + resolved = await resolve_orchestrator_gateway( + _FakeAsyncSession(config), + user_id="user-1", + organization_id="org-acme", + ) + assert resolved is None diff --git a/backend/tests/test_tenant_config_model.py b/backend/tests/test_tenant_config_model.py index ba263fb32..65d1985af 100644 --- a/backend/tests/test_tenant_config_model.py +++ b/backend/tests/test_tenant_config_model.py @@ -3,8 +3,9 @@ from pydantic import SecretStr from sqlalchemy import create_engine, text from sqlalchemy.orm import Session -from db.models import EncryptedString, TenantConfig, get_encryption_keyring, get_fernet from core.config import settings +from core.runtime_secrets import EncryptionConfigurationError +from db.models import EncryptedString, TenantConfig, get_encryption_keyring, get_fernet TEST_OPENAI_KEY = "test_key2" # noqa: S105 TEST_IMAP_PASSWORD = "imap-secret" # noqa: S105 @@ -79,7 +80,7 @@ def test_tenant_config_allows_same_user_in_different_organizations(db_session): def test_get_fernet_requires_encryption_key_even_when_debug_enabled(): settings.ENCRYPTION_KEY = None - with pytest.raises(RuntimeError, match="ENCRYPTION_KEY is required"): + with pytest.raises(EncryptionConfigurationError, match="ENCRYPTION_KEY is required"): get_fernet() diff --git a/docs/adr/0005-signed-noema-gateway-settings.md b/docs/adr/0005-signed-noema-gateway-settings.md new file mode 100644 index 000000000..88d342f93 --- /dev/null +++ b/docs/adr/0005-signed-noema-gateway-settings.md @@ -0,0 +1,50 @@ +# ADR-0005: Signed-session Noema gateway settings + +- **Status:** Proposed +- **Date:** 2026-08-20 +- **Scope:** Naruon's per-user Noema gateway settings API +- **Figma:** Not applicable; this slice is a backend control-plane contract and + adds no visual surface. + +## Context + +ADR-0004's Noema runtime path reads a dedicated gateway URL and Fernet-protected +token from the scoped `tenant_configs` record, but operators had no product API +to configure those values. Adding the fields to the mailbox self-service schema +would blur credential ownership and make a future frontend send unrelated mail +settings together with an inference credential. + +## Decision + +Naruon exposes `GET` and `PUT /api/noema-gateway` for the authenticated signed +session's `(user_id, organization_id)` scope. The route: + +1. validates an HTTPS `/v1` URL through the existing allowlist and global-address + transport policy; +2. stores the gateway token through the existing `EncryptedString` Fernet KV; +3. returns only `base_url`, `configured`, and `has_token`; +4. writes generic `AuditLog` and `SecurityAuditEvent` records without token + values; and +5. preserves the existing single-alias contextual-orchestrator runtime contract. + +The route does not accept a target user, does not manage mailbox credentials, +does not read environment provider keys, and does not add an organization-wide +fallback that would change Noema's existing per-user scope. A frontend must +omit blank secret fields when preserving a stored token. + +## Consequences + +Users can complete the gateway setup from a signed-session settings surface, +and operators can distinguish unconfigured gateway state without seeing a +credential. Organization-wide administration and frontend presentation remain +separate follow-up decisions because they require an explicit membership and +delegation contract. + +## Verification + +`backend/tests/test_noema_config_api.py` covers readiness responses, token +non-disclosure, audit records, URL rejection, empty updates, and extra-field +rejection. The focused Noema suite must pass with warnings treated as errors. + +The control mapping and APA 7th references are maintained in +[`docs/doctoring/noema-gateway-settings.md`](../doctoring/noema-gateway-settings.md). diff --git a/docs/adr/README.md b/docs/adr/README.md index 4d461fff6..f263bcdf8 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -13,6 +13,7 @@ govern implementation. | [ADR-0002](0002-fitted-topic-artifact-consumption.md) | Conditionally consume only a versioned fitted topic artifact through a fail-closed adapter | Proposed | Target `PLANNED`; runtime `BLOCKED-UPSTREAM` | | [ADR-0003](0003-separate-topic-measurement-from-agenda-generation.md) | Keep statistical measurement separate from agenda generation | Proposed | Target and future capability `PLANNED`; no implementation authorization | | [ADR-0004](0004-status-weighted-calendar-conflicts.md) | Evaluate CalDAV VEVENT overlaps by occupying status; cancelled does not occupy | Accepted | `ACCEPTED-NARUON-POLICY`; advisory evaluate API only | +| [ADR-0005](0005-signed-noema-gateway-settings.md) | Configure the scoped Noema gateway through a signed, audited, token-masking API | Proposed | Product setup path for the contextual-orchestrator runtime slice | The complete topic-intelligence requirements, architecture, contract, UML, conceptual ERD, security, test, and operability graph is indexed at diff --git a/docs/architecture/noema-decision-agent.md b/docs/architecture/noema-decision-agent.md new file mode 100644 index 000000000..abb941482 --- /dev/null +++ b/docs/architecture/noema-decision-agent.md @@ -0,0 +1,77 @@ +# Noema orchestrator routing + +Status: first slice implemented. `run_noema_agent` sends completions only to +contextual-orchestrator. Catalog files stay catalog-only. A Decision Points +/ `mail.triage` dispatcher is a later slice and is not in this change. + +The runnable contract lives in +[`backend/services/noema_agent.py`](../../backend/services/noema_agent.py) and +[`backend/services/orchestrator_gateway.py`](../../backend/services/orchestrator_gateway.py). + +## Problem + +naruon already registered a Noema stub (`registered_agents.json` → +`run_noema_agent`) that called the tenant LLM provider directly through +`resolve_runtime_llm_provider`. That path used a tenant `OpenAIChatModel` +(default `gpt-4o`) and left model choice inside naruon. The owner +requirement is the opposite: **model selection must go through +contextual-orchestrator**. + +`run_noema_agent` has no production callers. `registered_agents.json` and +`task_agent_mapping.json` are catalog only. Wiring Decision Points first +would lock the wrong picker. + +## Design + +naruon is a consumer. It does not reimplement the orchestrator catalog, Fugu / +Conductor / TRINITY selection, or list-price bookkeeping for free-but-priced +models. Those stay in ContextualWisdomLab/contextual-orchestrator. + +Noema sends OpenAI-compatible chat requests through one configured gateway: + +* a dedicated gateway inference token from the Fernet tenant KV + (`tenant_configs.noema_orchestrator_token`) +* an HTTPS base URL that ends in `/v1` + (`tenant_configs.noema_orchestrator_base_url`) +* the model alias `contextual-orchestrator` for each gateway request + +There is no sequential model list and no fail-over to the next agent or +provider inside naruon. Missing or rejected gateway config fails closed with +`error_code=orchestrator_gateway_unavailable`. + +This slice does **not** copy draft email-writing clients that still require a +tenant `model_profile_id`. Keep the existing tools, owner-scope, and opt-in +writeback surface. + +The setup contract is [`ADR-0005`](../adr/0005-signed-noema-gateway-settings.md): +`GET` and `PUT /api/noema-gateway` use the signed session's current owner scope, +validate the HTTPS allowlist, persist the token through Fernet, and return only +readiness metadata. Mailbox self-service fields and cross-user administration +are intentionally outside this route. + +Upstream org secrets (`NVIDIA_NIM_API_KEY`, `NVIDIA_NIM_API_KEY_SUB`, +`BYTEZ_API_KEY`, `OPENROUTER_API_KEY`, `OPENAI_API_KEY`) belong in the +orchestrator KV. naruon must not read them at request time. GitHub Models and +`COPILOT_GITHUB_TOKEN` are never a Noema path. OpenCode sidecar model lists +are not copied here. + +## Grounding + +Cost-aware routing belongs in a dedicated gateway, not in each caller +(Chen et al., 2023; Ong et al., 2024; Ding et al., 2024). naruon therefore +submits a single alias and lets contextual-orchestrator choose. + +## References + +Chen, L., Zaharia, M., & Zou, J. (2023). *FrugalGPT: How to use large language +models while reducing cost and improving performance*. arXiv. +https://doi.org/10.48550/arXiv.2305.05176 + +Ding, D., Mallick, A., Wang, C., Sim, R., Mukherjee, S., Rühle, V., +Lakshmanan, L. V. S., & Awadallah, A. H. (2024). *Hybrid LLM: Cost-efficient +and quality-aware query routing*. arXiv. +https://doi.org/10.48550/arXiv.2404.14618 + +Ong, I., Almahairi, A., Wu, V., Chiang, W.-L., Wu, T., Gonzalez, J. E., +Kadous, M. W., & Stoica, I. (2024). *RouteLLM: Learning to route LLMs with +preference data*. arXiv. https://doi.org/10.48550/arXiv.2406.18665 diff --git a/docs/doctoring/noema-gateway-settings.md b/docs/doctoring/noema-gateway-settings.md new file mode 100644 index 000000000..2e5e132af --- /dev/null +++ b/docs/doctoring/noema-gateway-settings.md @@ -0,0 +1,43 @@ +# Doctoring: Noema gateway settings + +## Change under review + +The signed-session `GET`/`PUT /api/noema-gateway` route configures the existing +per-user contextual-orchestrator gateway without exposing the gateway token. +It is a control-plane settings surface, not a mailbox credential surface and +not an organization-wide delegation API. + +## Evidence-backed controls + +| Control | Implementation evidence | Customer/operational action | +|---|---|---| +| Authenticated subject boundary | The route depends on the verified `AuthContext` and resolves `TenantConfig` with both `user_id` and `organization_id`. | Sign in to the intended organization before saving the gateway; do not reuse a token across organizations. | +| Credential confidentiality | `noema_orchestrator_token` uses the existing `EncryptedString` Fernet type; responses expose only `has_token`, and audit text is generic. | Confirm readiness from `has_token`; never paste the gateway token into support tickets or logs. | +| Endpoint validation | The URL must be HTTPS, end in `/v1`, pass the existing host allowlist, and resolve to global addresses. | Add the gateway host to the approved allowlist before saving it; a rejected URL is an actionable setup error. | +| Accountability | Successful updates create both `AuditLog` and `SecurityAuditEvent` records with a stable opaque resource UID and no token value. | Use the security audit surface to verify who changed the gateway and when. | +| Fail-closed behavior | Missing URL/token, malformed URL, control characters, invalid fields, and encryption-root failures return controlled errors; runtime resolution already fails closed. | Resolve the returned setup error before retrying Noema; do not bypass validation with environment keys. | + +These controls are aligned with the OWASP Application Security Verification +Standard's use as a verification baseline for web application security controls +(OWASP Foundation, 2025) and with NIST's current authentication and authenticator +management guidance (National Institute of Standards and Technology, 2025). This is an implementation mapping, +not a claim that Naruon is certified or conforms to every requirement in either +publication. + +## Test evidence + +`backend/tests/test_noema_config_api.py` covers ready/unready state, malformed +and unallowlisted URLs, omitted/masked/null token preservation, control +characters, extra fields, no-setting updates, token non-disclosure, generic +audit content, encryption-root failure, and unexpected database errors. The +module reaches 100% line coverage under the focused coverage command. The full +backend suite passes with warnings treated as errors. + +## References (APA 7th) + +National Institute of Standards and Technology. (2025, July). *Digital identity +guidelines: Authentication and authenticator management* (NIST Special +Publication 800-63B-4). https://doi.org/10.6028/NIST.SP.800-63B-4 + +OWASP Foundation. (2025). *OWASP Application Security Verification Standard +5.0.0*. https://owasp.org/www-project-application-security-verification-standard/ diff --git a/docs/papers/README.md b/docs/papers/README.md index 94437d5fb..84f4828c0 100644 --- a/docs/papers/README.md +++ b/docs/papers/README.md @@ -20,11 +20,14 @@ threads, and this literature grounds that requirements-engineering work. ## LLM cost, routing, and load balancing -Background for routing batch-tolerant embedding work through -**contextual-orchestrator** (the routing / cost hub) instead of calling a batch -engine directly. The orchestrator owns provider selection, load balancing, and -cost accounting; naruon submits a batch and records the reported cost. See -[`backend/services/batch_embedding_service.py`](../../backend/services/batch_embedding_service.py). +Background for routing batch-tolerant embedding work and in-process Noema +completions through **contextual-orchestrator** (the routing / cost hub) instead +of calling a provider or batch engine directly. The orchestrator owns provider +selection, load balancing, and cost accounting; naruon submits a batch or a +single `contextual-orchestrator` chat alias and records the reported cost. See +[`backend/services/batch_embedding_service.py`](../../backend/services/batch_embedding_service.py) +and +[`docs/architecture/noema-decision-agent.md`](../architecture/noema-decision-agent.md). - **`frugalgpt.pdf`** — L. Chen, M. Zaharia, J. Zou, *"FrugalGPT: How to Use Large Language Models diff --git a/registered_agents.json b/registered_agents.json index 312a0488d..434c29e72 100644 --- a/registered_agents.json +++ b/registered_agents.json @@ -5,7 +5,7 @@ "framework": "pydantic-ai", "framework_license": "MIT", "entrypoint": "services.noema_agent:run_noema_agent", - "description": "General-purpose assistant that reasons over the naruon workspace (mail, content graph, tasks) on the tenant's configured LLM provider and dispatches opt-in, audit-logged writebacks through the self-hosted runner.", + "description": "General-purpose assistant that reasons over the naruon workspace (mail, content graph, tasks) through the contextual-orchestrator gateway and dispatches opt-in, audit-logged writebacks through the self-hosted runner. naruon does not pick a tenant chat model, hold upstream provider keys, or sequentially fail over models.", "capabilities": [ "mail.search", "mail.read", @@ -14,7 +14,9 @@ "tasks.update", "calendar.writeback" ], - "provider_source": "runtime_llm_provider", + "provider_source": "contextual-orchestrator", + "model_alias": "contextual-orchestrator", + "sequential_failover": false, "writeback": { "opt_in": true, "audit_logged": true