fix(tools): remove lexical pseudo-topic models

AGENTS.md

@@ -95,6 +95,17 @@ in this repo.
   knowledge-graph pipeline (DOM decomposition, entity/relation extraction,
   grounded graph retrieval) should ground itself in the relevant layout-analysis
   and knowledge-graph / grounded-retrieval literature.
+
+### Structural topic-model boundary
+
+- Do not implement or describe hard-coded term lists, term frequency,
+  embeddings, or LLM-assigned labels as structural topic modeling (STM).
+  Fixed business labels are not topic-posterior estimates, and the explicitly
+  lexical `keyword_extractor` must not be used as topic evidence.
+- Topic inference requires a versioned fitted TEPP model and its frozen
+  preprocessing and vocabulary contract. If that fitted model is unavailable,
+  fail closed; do not return a default label, template agenda, or substitute
+  keyword/embedding/LLM result presented as STM.
 <!-- END cwl-agent-guidance -->
 
 ## Release governance defaults

CHANGELOG.md

@@ -1,4 +1,13 @@
 ## [Unreleased]
+### 주제 측정 경계 (Topic Measurement)
+
+- STM 결과로 오인될 수 있었던 하드코딩 용어표 기반
+  `email_categorizer`와 `meeting_agenda_generator`를 도구 레지스트리에서
+  제거했습니다. `keyword_extractor`는 결정론적 단어 빈도 유틸리티로 유지하되
+  주제 posterior 근거로 사용하지 않는 경계를 문서화했습니다. 현재 Naruon에는
+  fitted TEPP 모델 기반 production 주제 측정 API가 없으므로, 모델 부재 시
+  기본 라벨이나 템플릿으로 대체하지 않고 fail closed 합니다.
+
 ### 보안 패치 (CodeQL extended current-head)
 
 - `cryptography`를 `50.0.0`으로 갱신해 공격자 제공 PKCS#7 EnvelopedData 복호화 결과의 오류·타이밍 차이로 발생하는 Bleichenbacher oracle(`CVE-2026-69247`, `GHSA-g6cj-pr64-35w5`)을 제거하고, backend·uv lock·hash lock·Strix CI 의존성 증거를 같은 버전으로 동기화했습니다. Strix 잠금은 `google-cloud-aiplatform==1.160.0`의 `<7` 제약을 위반하던 `protobuf==7.35.1`을 이미 검증된 `6.33.6`으로 복구해 다시 해석·설치 가능하게 했습니다.

backend/api/tools.py

@@ -686,26 +686,6 @@ async def base64_decoder_handler(params: Dict[str, Any]) -> Dict[str, str]:
         "합니다",
     }
 )
-_CATEGORY_TERMS = (
-    ("Urgent", ("urgent", "asap", "immediate", "긴급", "시급", "빨리")),
-    ("Finance", ("invoice", "billing", "payment", "결제", "청구", "송금")),
-    ("Scheduling", ("meeting", "schedule", "appointment", "회의", "일정", "약속")),
-)
-_AGENDA_TOPICS = (
-    ("Project Status Update", ("project", "프로젝트", "과제")),
-    ("Discuss Pending Issues", ("issue", "bug", "blocker", "문제", "오류", "장애")),
-    ("Decisions Required", ("decision", "approve", "결정", "승인")),
-    (
-        "Timeline and Milestones",
-        ("deadline", "milestone", "timeline", "마감", "기한", "일정"),
-    ),
-    (
-        "Budget and Resource Review",
-        ("budget", "cost", "resource", "예산", "비용", "자원"),
-    ),
-)
-
-
 def _normalize_analysis_text(value: str) -> str:
     """Normalize user text for deterministic, multilingual rule matching."""
     if len(value) > ANALYSIS_TEXT_MAX_CHARS:
@@ -720,44 +700,8 @@ def _analysis_tokens(value: str) -> list[str]:
     return _ANALYSIS_TOKEN_PATTERN.findall(_normalize_analysis_text(value))
 
 
-def _contains_analysis_term(normalized_text: str, term: str) -> bool:
-    """Match ASCII terms on word boundaries and Korean terms as morpheme stems."""
-    normalized_term = _normalize_analysis_text(term)
-    if normalized_term.isascii():
-        pattern = rf"(?<![a-z0-9]){re.escape(normalized_term)}(?![a-z0-9])"
-        return re.search(pattern, normalized_text) is not None
-    return normalized_term in normalized_text
-
-
-async def email_categorizer_handler(params: Dict[str, Any]) -> Any:
-    """Categorize email text with deterministic Korean and English rules."""
-    content = _normalize_analysis_text(params.get("email_content", ""))
-    categories = [
-        category
-        for category, terms in _CATEGORY_TERMS
-        if any(_contains_analysis_term(content, term) for term in terms)
-    ]
-
-    if not categories:
-        categories = ["General"]
-
-    return {"categories": categories, "primary_category": categories[0]}
-
-
-registry.register(
-    ToolInfo(
-        code="email_categorizer",
-        name="이메일 자동 분류기 (Email Categorizer)",
-        description="이메일 내용을 분석하여 알맞은 카테고리로 자동 분류합니다.",
-        category="이메일 분석",
-        parameters={"email_content": "string"},
-    ),
-    email_categorizer_handler,
-)
-
-
 async def keyword_extractor_handler(params: Dict[str, Any]) -> Any:
-    """Extract stable keywords ranked by frequency and first occurrence."""
+    """Extract deterministic lexical terms by frequency and first occurrence."""
     candidates = [
         token
         for token in _analysis_tokens(params.get("text", ""))
@@ -781,46 +725,14 @@ async def keyword_extractor_handler(params: Dict[str, Any]) -> Any:
     ToolInfo(
         code="keyword_extractor",
         name="주요 키워드 추출기 (Keyword Extractor)",
-        description="텍스트 본문에서 가장 중요한 키워드를 추출합니다.",
+        description="텍스트 본문에서 빈도와 최초 출현 순으로 반복 용어를 추출합니다.",
         category="이메일 분석",
         parameters={"text": "string"},
     ),
     keyword_extractor_handler,
 )
 
 
-async def meeting_agenda_generator_handler(params: Dict[str, Any]) -> Any:
-    """Generate a deterministic agenda from Korean or English discussion topics."""
-    context = _normalize_analysis_text(params.get("discussion_context", ""))
-    if len(_analysis_tokens(context)) < 2:
-        return {
-            "agenda_items": ["Introductions", "Open Discussion"],
-            "estimated_duration_minutes": 30,
-        }
-
-    items = ["Review previous action items"]
-    items.extend(
-        agenda_item
-        for agenda_item, terms in _AGENDA_TOPICS
-        if any(_contains_analysis_term(context, term) for term in terms)
-    )
-    items.append("Next Steps and Action Items")
-
-    return {"agenda_items": items, "estimated_duration_minutes": len(items) * 15}
-
-
-registry.register(
-    ToolInfo(
-        code="meeting_agenda_generator",
-        name="회의 아젠다 생성기 (Meeting Agenda Generator)",
-        description="논의 컨텍스트를 바탕으로 적절한 회의 아젠다를 자동으로 생성합니다.",
-        category="일정 관리",
-        parameters={"discussion_context": "string"},
-    ),
-    meeting_agenda_generator_handler,
-)
-
-
 @router.get("/tools", response_model=list[ToolInfo])
 def get_tools() -> list[ToolInfo]:
     """

backend/tests/test_tools_api.py

@@ -112,6 +112,19 @@ def test_get_tool_not_found():
     assert response.json() == {"detail": "Tool not found"}
 
 
+@pytest.mark.parametrize(
+    "tool_code", ["email_categorizer", "meeting_agenda_generator"]
+)
+def test_registry_omits_lexical_pseudo_topic_tools(tool_code):
+    assert registry.get(tool_code) is None
+
+
+def test_keyword_extractor_is_disclosed_as_lexical_term_frequency():
+    tool = registry.get("keyword_extractor")
+    assert tool is not None
+    assert "빈도" in tool.description
+
+
 @pytest.mark.asyncio
 async def test_execute_tool_success():
     with TestClient(app) as client:
@@ -1130,52 +1143,6 @@ def test_detect_text_language_ko():
     assert _detect_text_language("안녕하세요") == "ko"
 
 
-@pytest.mark.asyncio
-async def test_email_categorizer_handler():
-    from api.tools import email_categorizer_handler
-
-    # Test Finance category
-    result = await email_categorizer_handler(
-        {"email_content": "Please pay this invoice soon."}
-    )
-    assert "Finance" in result["categories"]
-
-    # Test Scheduling category
-    result = await email_categorizer_handler(
-        {"email_content": "Let's schedule a meeting."}
-    )
-    assert "Scheduling" in result["categories"]
-
-    # Test Urgent category
-    result = await email_categorizer_handler({"email_content": "This is urgent!"})
-    assert "Urgent" in result["categories"]
-
-    # Test General category (fallback)
-    result = await email_categorizer_handler({"email_content": "Hello, how are you?"})
-    assert "General" in result["categories"]
-
-    # Test multiple categories
-    result = await email_categorizer_handler(
-        {"email_content": "URGENT: Meeting to discuss invoice payment"}
-    )
-    assert result == {
-        "categories": ["Urgent", "Finance", "Scheduling"],
-        "primary_category": "Urgent",
-    }
-
-    # ASCII category rules use token boundaries instead of substring matching.
-    result = await email_categorizer_handler(
-        {"email_content": "The prepayment plan is documented."}
-    )
-    assert result["categories"] == ["General"]
-
-    # Unicode compatibility forms and Korean stems remain matchable.
-    result = await email_categorizer_handler(
-        {"email_content": "긴급 회의에서 청구 금액을 검토합니다."}
-    )
-    assert result["categories"] == ["Urgent", "Finance", "Scheduling"]
-
-
 @pytest.mark.asyncio
 async def test_keyword_extractor_handler():
     from api.tools import keyword_extractor_handler
@@ -1199,41 +1166,6 @@ async def test_keyword_extractor_handler():
     assert empty == {"keywords": [], "keyword_count": 0}
 
 
-@pytest.mark.asyncio
-async def test_meeting_agenda_generator_handler():
-    from api.tools import meeting_agenda_generator_handler
-
-    # Test with short context
-    result = await meeting_agenda_generator_handler({"discussion_context": "short"})
-    assert result["agenda_items"] == ["Introductions", "Open Discussion"]
-    assert result["estimated_duration_minutes"] == 30
-
-    # Test with project and issue context
-    result = await meeting_agenda_generator_handler(
-        {"discussion_context": "The project has an issue that needs fixing."}
-    )
-    assert "Review previous action items" in result["agenda_items"]
-    assert "Project Status Update" in result["agenda_items"]
-    assert "Discuss Pending Issues" in result["agenda_items"]
-    assert "Next Steps and Action Items" in result["agenda_items"]
-    assert result["estimated_duration_minutes"] == len(result["agenda_items"]) * 15
-
-    # Korean context covers decision, timeline, and resource agenda paths.
-    result = await meeting_agenda_generator_handler(
-        {"discussion_context": "프로젝트 예산 승인과 마감 일정 문제를 결정합니다."}
-    )
-    assert result["agenda_items"] == [
-        "Review previous action items",
-        "Project Status Update",
-        "Discuss Pending Issues",
-        "Decisions Required",
-        "Timeline and Milestones",
-        "Budget and Resource Review",
-        "Next Steps and Action Items",
-    ]
-    assert result["estimated_duration_minutes"] == 105
-
-
 def test_execute_analysis_tool_rejects_oversized_text():
     from api.tools import ANALYSIS_TEXT_MAX_CHARS
 

docs/adr/0001-topic-measurement-authority.md

@@ -0,0 +1,63 @@
+# ADR-0001: Structural topic measurement is a TEPP model-artifact boundary
+
+**Status:** Accepted  
+**Date:** 2026-08-09
+
+## Context
+
+Naruon historically exposed `email_categorizer` and `meeting_agenda_generator` from small hard-coded Korean/English term tables. Those outputs were deterministic, but they were lexical rules presented through product names that implied semantic topic inference. That is not a Structural Topic Model and provides no fitted corpus-level topic identity, mixed-membership posterior, uncertainty, prevalence/content covariate effect, multilingual measurement evidence, or model-artifact provenance.
+
+The CWL scientific boundary is already defined in TEPP: LLM-assisted multilingual semantic evidence may support measurement, while fitted statistical topic inference remains a versioned Rust-first model with explicit preprocessing/vocabulary, temporal availability and multilevel/multiple-membership structure where applicable. Naruon is a consuming workspace/product surface, not a second independent topic-estimation authority.
+
+## Decision
+
+1. Naruon's retained `keyword_extractor` remains explicitly lexical metadata only. It must never be described as a topic model or semantic classifier.
+2. Naruon will not replace removed pseudo-topic tools with a larger keyword table, embedding cluster, zero-shot labeler, or LLM prompt while naming the result Structural Topic Modeling.
+3. Production topic inference, when available, must consume a **versioned fitted TEPP model artifact** through a stable typed integration boundary. Naruon must not refit an STM per request.
+4. A valid inference request/result must bind at least: model artifact/version and digest; immutable source/document identity; frozen preprocessing and vocabulary; OOV/retained-token diagnostics; language profile/support status; relevant prevalence/content and multilevel/cross-classified/multiple-membership covariates; event/document/availability/knowledge-cutoff time semantics when the model uses them; mixed-membership topic proportions; posterior uncertainty/diagnostics; and explicit abstention/failure status.
+5. Human-readable topic labels and generated agenda/action summaries are presentation/generation artifacts. They are never the numeric topic identity and cannot change the fitted posterior.
+6. If the required TEPP model/API/artifact is unavailable, incompatible, under-supported for the document language, or cannot produce an evidence-valid posterior, Naruon fails closed. It does not fabricate `General`, empty agenda semantics, or an embedding/LLM substitute under the same contract.
+7. TEPP remains independently operable and Naruon remains independently useful without topic inference. Integration is optional and versioned; Naruon must not read TEPP's private database directly.
+
+## Alternatives rejected
+
+### Keep deterministic keyword categories
+
+Rejected because deterministic lexical matching is not mixed-membership topic measurement and would preserve the original product-truth defect.
+
+### Use embeddings or clustering as a drop-in STM replacement
+
+Rejected as a semantic product substitution. Such methods may be useful in separate features, but equal semantic usefulness does not make them an STM posterior or preserve the same prevalence/content/uncertainty contract.
+
+### Ask an LLM for topic labels at request time
+
+Rejected as the statistical authority. LLMs may interpret or label fitted evidence behind a separate bounded contract, but request-time labels do not replace a fitted corpus-level model and its uncertainty.
+
+### Fit a fresh topic model for every Naruon request
+
+Rejected because new-document inference must be comparable against a stable fitted model. Per-request refits destroy topic identity, reproducibility, governance, and longitudinal comparability.
+
+## Consequences
+
+- PR #1297 removes the misleading pseudo-topic tools rather than shipping an unvalidated replacement.
+- The next topic-related product work belongs first in TEPP: production fitted-model artifact and inference API, realistic model validation, then a Naruon adapter.
+- Naruon tests must keep lexical utilities labelled lexical and must fail if removed pseudo-topic registry entries reappear without an accepted replacement contract.
+- Any future adapter must carry model/provenance/uncertainty/diagnostic fields rather than only a label string.
+- Product documentation must distinguish `implemented on protected develop`, `active PR`, and `accepted target`; this ADR does not claim TEPP topic inference exists today.
+
+## Verification / acceptance
+
+Before a future topic-measurement adapter is promoted to protected `develop`, require:
+
+- TEPP production artifact/inference API available at a versioned contract;
+- fitted-model and preprocessing/vocabulary identity validation;
+- positive, negative, OOV/insufficient-text, unsupported-language and model-unavailable tests;
+- posterior normalization and uncertainty/diagnostic tests;
+- multilevel/multiple-membership and temporal-covariate contract tests when those inputs are part of the fitted model;
+- tenant/source authorization at the Naruon boundary;
+- exact-head CI/security/coverage and independent review;
+- no claim that topic labels or LLM interpretations are the numeric topic identity.
+
+## Supersession rule
+
+Changing the statistical authority away from TEPP, changing new-document topic identity semantics, or authorizing Naruon to fit its own production topic models requires a superseding ADR plus synchronized product/technical/architecture/test/operability documentation and scientific validation evidence.

docs/adr/README.md

@@ -0,0 +1,13 @@
+# Naruon Architecture Decision Records
+
+This index records cross-cutting decisions that must survive beyond an individual pull request, implementation plan, or chat. `Accepted` means the decision governs architecture; it does not mean a future integration described by the ADR is already implemented on protected `develop`.
+
+| ADR | Decision | Status |
+|---|---|---|
+| [ADR-0001](0001-topic-measurement-authority.md) | Structural topic measurement is a versioned TEPP model-artifact boundary, never a keyword/label heuristic | Accepted |
+
+## Change rule
+
+Create or update an ADR when a change moves product authority between Naruon and another CWL service, introduces a new scientific/statistical inference contract, changes persistence or tenant authority, changes model/credential trust boundaries, or replaces a fail-closed product capability with a different production owner.
+
+Every implementing PR must keep the corresponding source, tests, doctoring, architecture/operability contract, and CHANGELOG maturity truthful. An active PR or accepted target must not be described as protected-branch implementation before it is integrated and independently verified.