[twschiller]

Summary

• Remove the Category browse: /{country}/{lang}/cat/{slug}-{categoryCode}/ line from extension/data/sites/ikea.yaml. {categoryCode} is a host-internal code the agent can't read off the page or be told at runtime; agents were inventing values (billy-bookcases-18732, billy-bookcases-16282) that redirected to unrelated categories and then recovering via search anyway. The Search: and Direct product: templates remain — those compose cleanly from runtime intent.
• Pin the ikea-billy-cheapest-white benchmark task to the standard shortest BILLY bookcase variant in white (excluding BILLY OXBERG, height-extension, and accessory units) so the "smallest white BILLY variant" is no longer ambiguous between bookcases, extensions, and shelves.
• Add a "Composability rule for search-url-helper templates" subsection to skills/agent-browser-shield-site-rules/SKILL.md so future recipes don't reintroduce templates requiring identifiers the agent can't plausibly read or be told. The rule explicitly warns against deriving recipes from benchmark/tasks.csv — recipes must reflect general agent information needs, not benchmark ground truth.

Evidence

run	baseline	guarded	notes
run_20260601_204107_e2ee	6.7 steps · $0.044	11.3 steps · $0.104	guarded r1/r2 invented category codes
cmp_20260602_030512_dea7 (before fix)	17.3 steps · $0.183	10.0 steps · $0.073	guarded r3 hallucinated billy-bookcases-18732
cmp_20260602_031552_4d23 (after fix)	15.0 steps · $0.149	16.0 steps · $0.164	0 goto /cat/... attempts across all 6 reps

Aggregate guarded vs baseline is dominated by n=3 stochasticity and an inconsistent gpt-4o-mini judge (the same correctly-formatted answer was marked PASS in one run and FAIL in another). The pinned task spec should reduce future judge variance independently of the rule change.

Test plan

• cd extension && bun run check passes
• cd extension && bun run build regenerates site-data.generated.ts cleanly (already committed)
• On a follow-up benchmark run, no rep on ikea-billy-cheapest-white issues a goto to /us/en/cat/...
• Spot-check that other IKEA-bound tasks still benefit from the Search: URL hint (e.g., a fresh compare_scenarios.py rep that uses goto /search/?q=... in step 0–2)

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
agent-browser-shield-demo-site	Ready	Preview, Comment	Jun 2, 2026 3:30am