Skip to content
Draft
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
7359d52
feat: URL 추출기 (url_extractor) 도구 추가
seonghobae Aug 31, 2026
750c4b1
feat: URL 추출기 (url_extractor) 도구 추가
seonghobae Aug 31, 2026
0584656
fix: URL regex pattern trailing period handling without lookbehinds
seonghobae Aug 31, 2026
8db683d
fix(tools): preserve bounded complete URLs
seonghobae Aug 31, 2026
5e220ef
fix(tools): bound URL delimiter cleanup
seonghobae Aug 31, 2026
687f472
fix(tools): require wrapper evidence for trimming
seonghobae Aug 31, 2026
9c5bc0f
fix(tools): handle spaced URL wrappers
seonghobae Aug 31, 2026
e0209af
fix(tools): preserve balanced URL delimiters
seonghobae Aug 31, 2026
f4710ca
fix: Update URL extraction tool to correctly parse delimited URLs wit…
seonghobae Sep 1, 2026
a83f237
fix: Preserve valid balanced URL delimiters
seonghobae Sep 1, 2026
23ef83e
fix(tools): preserve balanced URL delimiters
seonghobae Sep 3, 2026
49bbfa4
merge(stack): place URL extractor on tool owner
seonghobae Sep 4, 2026
5135a21
URL 추출기 도구 추가 및 관련 CI/보안/버그 수정
seonghobae Sep 4, 2026
260db8f
merge: restack url extractor on current tool base
seonghobae Sep 4, 2026
dc9ff84
Merge remote-tracking branch 'origin/feature/url-extractor-tool-13801…
seonghobae Sep 5, 2026
542c1b9
Merge remote-tracking branch 'origin/feature/new-analysis-tools-68409…
seonghobae Sep 5, 2026
2e25b16
Merge remote-tracking branch 'origin/feature/new-analysis-tools-68409…
seonghobae Sep 5, 2026
1bdb0c7
URL 추출기 도구 추가 및 관련 CI/보안/버그 수정
seonghobae Sep 5, 2026
6e37926
Merge remote-tracking branch 'origin/feature/new-analysis-tools-68409…
seonghobae Sep 5, 2026
14e4630
Revert "URL 추출기 도구 추가 및 관련 CI/보안/버그 수정"
seonghobae Sep 5, 2026
4a465d3
fix(tools): isolate URL extractor delta
seonghobae Sep 5, 2026
5014fb3
Merge remote-tracking branch 'origin/feature/new-analysis-tools-68409…
seonghobae Sep 5, 2026
a1ffc7e
URL 추출기 도구 추가 및 관련 CI/보안/버그 수정
seonghobae Sep 5, 2026
46ff31d
merge(concurrency): preserve validated URL extractor tree
seonghobae Sep 5, 2026
d6f8db1
URL 추출기 도구 추가 및 관련 CI/보안/버그 수정
seonghobae Sep 5, 2026
7df1823
merge(concurrency): preserve validated URL extractor tree
seonghobae Sep 5, 2026
a0e877e
merge(stack): inherit repaired matcher owner head
seonghobae Sep 5, 2026
977bfca
URL 추출기 도구 추가 및 관련 CI/보안/버그 수정
seonghobae Sep 5, 2026
cc9591d
merge(concurrency): retain validated URL extractor after agent rewrite
seonghobae Sep 5, 2026
17b732a
merge(stack): inherit current matcher owner head
seonghobae Sep 5, 2026
f1c7f6d
merge(tools): restack URL extractor on canonical matcher owner
seonghobae Sep 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
## [Unreleased]
- 텍스트 본문에서 HTTP 및 HTTPS URL을 추출하여 중복 없이 반환하는 유틸리티 도구인 `url_extractor` (URL 추출기)를 추가했습니다.
- 긴 이메일·첨부 본문을 의미 단위 청크로 임베딩한 뒤 기존 email/attachment 벡터 계약으로 평균화하고, 청크 요청·벡터 누적을 제한된 창으로 처리합니다. OpenAI `text-embedding-3-*`에는 저장 차원(`1536`)을 직접 요청하도록 보강했습니다. 합성 메일 fixture 5건(70청크)과 provider 요청 계약으로 1,536차원 벡터 경로를 검증했으며, 실행 시 선택한 임베딩 제공자에 본문·파싱된 첨부 텍스트를 전송할 수 있습니다. 회사 기밀 데이터는 fixture·commit·PR·log에 포함하지 않습니다.
- EmailDetail 테스트가 지원하지 않는 스레드 병합/분리 버튼을 `textContent`뿐 아니라 `aria-label`과 `title` 접근 가능 이름으로도 검출하도록 바꿔, 아이콘 전용 버튼 회귀를 놓치지 않습니다.

Expand Down
62 changes: 62 additions & 0 deletions backend/api/tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -769,6 +769,68 @@ async def uuid_v4_generator_handler(params: Dict[str, Any]) -> Dict[str, str]:
)


_URL_PATTERN = re.compile(r"https?://[^\s<>\"']+", re.IGNORECASE)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Stop matches before a second URL scheme

When URLs are adjacent without whitespace, this greedy character class consumes the later scheme as part of the first candidate. For example, Markdown such as [https://a.example](https://b.example) becomes one candidate; the nearby urlsplit validation rejects that candidate, so both URLs are omitted. Stop at Markdown delimiters or restart matching at an embedded HTTP(S) scheme, and cover this case in backend/tests/test_tools_api.py.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat typographic quotes as URL boundaries

In rich-text email containing a smart-quoted link such as “https://example.com”, the pattern stops only at ASCII quotes and therefore returns https://example.com”. urlsplit still supplies a hostname, so the malformed candidate is reported as a successful extraction; add common Unicode opening/closing quotation marks to the prose boundaries and a focused regression test.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Pass the regex flag by keyword

This compile call supplies re.IGNORECASE positionally, contrary to the repository-wide requirement that standard-library regex flags use the explicit flags= keyword. Change it to re.compile(..., flags=re.IGNORECASE) so the new extractor follows the mandated warning-resistant regex convention.

AGENTS.md reference: AGENTS.md:L687-L687

Useful? React with 👍 / 👎.

_PROSE_TRAILING_PUNCTUATION = ".,;:!?"


def _trim_url_candidate(candidate: str, wrapping_openers: str) -> str:
"""Remove only closing delimiters proven by adjacent opening wrappers."""
delimiters = (("(", ")"), ("[", "]"), ("{", "}"))
excess = {
closer: min(
wrapping_openers.count(opener),
max(0, candidate.count(closer) - candidate.count(opener)),
)
Comment thread
seonghobae marked this conversation as resolved.
for opener, closer in delimiters
}
without_prose = candidate.rstrip(_PROSE_TRAILING_PUNCTUATION)
end = len(without_prose)
while end and excess.get(without_prose[end - 1], 0):
excess[without_prose[end - 1]] -= 1
end -= 1
return without_prose[:end] if end < len(without_prose) else candidate
Comment thread
devin-ai-integration[bot] marked this conversation as resolved.

async def url_extractor_handler(params: Dict[str, Any]) -> Dict[str, list[str]]:
text = params["text"]
if len(text) > ANALYSIS_TEXT_MAX_CHARS:
raise ValueError(
f"Analysis text must not exceed {ANALYSIS_TEXT_MAX_CHARS} characters"
)
urls: list[str] = []
seen: set[str] = set()
for match in _URL_PATTERN.finditer(text):
wrapper_start = match.start()
while wrapper_start and text[wrapper_start - 1].isspace():
wrapper_start -= 1
wrapper_end = wrapper_start
Comment thread
devin-ai-integration[bot] marked this conversation as resolved.
while wrapper_start and text[wrapper_start - 1] in "([{":
wrapper_start -= 1
Comment thread
devin-ai-integration[bot] marked this conversation as resolved.
candidate = _trim_url_candidate(
match.group(), text[wrapper_start:wrapper_end]
)
Comment thread
seonghobae marked this conversation as resolved.
try:
parsed = urllib.parse.urlsplit(candidate)
_ = parsed.port # validate a declared port without requiring one
valid = parsed.hostname is not None
except ValueError:
valid = False
if valid and candidate not in seen:
seen.add(candidate)
urls.append(candidate)
return {"urls": urls}


registry.register(
ToolInfo(
code="url_extractor",
name="URL 추출기 (URL Extractor)",
description="텍스트 본문에서 HTTP 및 HTTPS URL을 추출합니다.",
category="유틸리티",
parameters={"text": "string"},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Collect actual text before exposing the catalog tool

Declaring the required text parameter automatically exposes this tool on the existing workspace tools page, but that page only displays parameter metadata and handleExecute substitutes the literal "test_value" for every string. A user therefore cannot supply source text, and clicking URL Extractor always submits a non-URL and returns an empty list; add an editable text value wired into the execute payload (and render the returned URLs), then cover it with corepack pnpm@11.5.3 --dir frontend exec vitest run src/app/tools/page.test.tsx.

AGENTS.md reference: AGENTS.md:L292-L293

Useful? React with 👍 / 👎.

),
url_extractor_handler,
)


@router.get("/tools", response_model=list[ToolInfo])
def get_tools() -> list[ToolInfo]:
Expand Down
162 changes: 162 additions & 0 deletions backend/tests/test_tools_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
os.environ.setdefault("AUTH_SESSION_HMAC_SECRET", secrets.token_urlsafe(48))

from api.tools import (
ANALYSIS_TEXT_MAX_CHARS,
MAX_TOOL_FAILURE_MESSAGE_CHARS,
ExecuteRequest,
ToolInfo,
Expand Down Expand Up @@ -381,6 +382,167 @@ async def error_handler(params):
assert "Simulated error" in data["message"]


def test_execute_url_extractor():
text = (
"URLs: https://example.com:8443/foo/bar?q=a%20b#section "
"(http://foo.com?next=/a) https://example.com:8443/foo/bar?q=a%20b#section."
)
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={"parameters": {"text": text}},
)
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
assert data["result"]["urls"] == [
"https://example.com:8443/foo/bar?q=a%20b#section",
"http://foo.com?next=/a",
"https://example.com:8443/foo/bar?q=a%20b#section.",
]


@pytest.mark.parametrize(
"url",
(
"https://example.com/a,b,",
"https://example.com?q=what?",
"https://example.com#frag!",
),
)
def test_execute_url_extractor_preserves_valid_terminal_punctuation(url):
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={"parameters": {"text": url}},
)

assert response.json()["result"]["urls"] == [url]


@pytest.mark.parametrize("suffix", (").", "),", ")!"))
def test_execute_url_extractor_removes_wrapped_prose_delimiters(suffix):
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={"parameters": {"text": f"(https://example.com/path{suffix}"}},
)

assert response.json()["result"]["urls"] == ["https://example.com/path"]


@pytest.mark.parametrize(
("text", "url"),
(
("(https://example.com/path!),", "https://example.com/path!"),
("([https://example.com/path]).", "https://example.com/path"),
),
)
def test_execute_url_extractor_handles_punctuated_nested_wrappers(text, url):
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={"parameters": {"text": text}},
)

assert response.json()["result"]["urls"] == [url]


def test_execute_url_extractor_handles_spaced_nested_wrappers():
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={"parameters": {"text": "([ https://example.com/path])."}},
)

assert response.json()["result"]["urls"] == ["https://example.com/path"]


def test_execute_url_extractor_does_not_borrow_distant_wrapper():
url = "https://example.com/path)."
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={"parameters": {"text": f"(see {url}"}},
)

assert response.json()["result"]["urls"] == [url]


@pytest.mark.parametrize(
"url",
(
"https://example.com/a(b)",
"https://example.com/a[b]",
"https://example.com/a{b}",
),
)
def test_execute_url_extractor_preserves_balanced_url_delimiters(url):
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={"parameters": {"text": f"(\n{url}"}},
)

assert response.json()["result"]["urls"] == [url]


@pytest.mark.parametrize(
"url",
(
"https://example.com/a).",
"https://example.com/a],",
"https://example.com/a}!",
),
)
def test_execute_url_extractor_preserves_unwrapped_delimiter_suffixes(url):
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={"parameters": {"text": url}},
)

assert response.json()["result"]["urls"] == [url]


def test_execute_url_extractor_handles_many_unmatched_delimiters_linearly():
wrapper_count = 25_000
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={
"parameters": {
"text": "(" * wrapper_count
+ "https://example.com/"
+ ")" * wrapper_count
}
},
)

assert response.json()["result"]["urls"] == ["https://example.com/"]


def test_execute_url_extractor_rejects_oversized_text():
with TestClient(app) as client:
response = client.post(
"/api/tools/url_extractor/execute",
headers={"Authorization": f"Bearer {_signed_session_token()}"},
json={"parameters": {"text": "x" * (ANALYSIS_TEXT_MAX_CHARS + 1)}},
)

assert response.status_code == 200
assert response.json()["status"] == "failed"


@pytest.mark.asyncio
async def test_execute_tool_failure_log_does_not_include_user_controlled_lines(caplog):
hostile_code = "error_tool\r\nforged_event=true"
Expand Down
Loading