Skip to content

Drop search-URL templates that rely on opaque host ids - #76

Merged
twschiller merged 1 commit into
mainfrom
crawl/audit-search-url-composability
Jun 2, 2026
Merged

twschiller merged 1 commit into
mainfrom
crawl/audit-search-url-composability

Conversation

@twschiller

Copy link
Copy Markdown
Contributor

Summary

Audited all search-url-helper recipes in extension/data/sites/*.yaml against the composability rule in skills/agent-browser-shield-site-rules/SKILL.md and applied the IKEA precedent (#75) across the rest of the catalog.

  • Dropped templates whose {variable} resolves to an opaque host-internal id (category codes, neighborhood/region ids, attr-pill ids, etc.) — 11 hosts: allrecipes, bestbuy, ebay, expedia, homedepot, hotels, indeed, lowes, newegg, opentable, tripadvisor. Inline comments explain each decision so the same shape isn't re-added.
  • Clarified 4 borderline hosts (autotrader, cars-com, steam, wayfair) where the id can be discovered from a prior search/redirect — added explicit "do not invent — discover via X" guard text rather than dropping the template.

Test plan

  • bun run build-site-data regenerates cleanly
  • bun run build builds the extension
  • bun run check (biome + eslint) passes
  • CI green

🤖 Generated with Claude Code

Audited all `search-url-helper` recipes against the composability rule in
skills/agent-browser-shield-site-rules/SKILL.md and dropped templates
whose {variable} resolves to an opaque host-internal id (category codes,
internal SKUs, region/geo ids) the agent has no way to read or be told —
the IKEA category-code failure mode from #75, repeated across 11 hosts.

For each dropped template, left an inline comment explaining the
decision so a future contributor doesn't re-add the same shape. For
borderline cases (autotrader/cars-com dealer ids, steam tag ids,
wayfair categoryId, bestbuy productId), kept the template but added an
explicit "do not invent — discover via X" guard sentence.

Affected: allrecipes, bestbuy, ebay, expedia, homedepot, hotels,
indeed, lowes, newegg, opentable, tripadvisor (drops);
autotrader, cars-com, steam, wayfair (clarifications).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-browser-shield-demo-site Ready Ready Preview, Comment Jun 2, 2026 3:54am

Request Review

@twschiller
twschiller merged commit 44053b2 into main Jun 2, 2026
7 checks passed
@twschiller
twschiller deleted the crawl/audit-search-url-composability branch June 2, 2026 03:56

This branch was successfully deployed

1 active deployment
Preview — 78fc4dd0 Deployed Jun 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant