Complete competitor watchlist sentiment ingestion

.github/agents/content-scout.agent.md

@@ -1245,6 +1245,23 @@ Where the platform exposes them, also include engagement metrics (likes/upvotes/
 |------------|---------------|-----------|-------------------|---------------|
 <!-- Switching signals: posts about migrating to/from this competitor -->
 
+<!--
+How to populate (when Competitor tracking is on): the competitors come from the config's `## Competitors`
+section. Match each competitor by name AND its listed aliases across the scanned conversation sources
+(Reddit / X / LinkedIn / Bluesky / HN / Stack Overflow) plus the competitor's own blog / release-notes.
+Reddit + X mentions come pre-matched from the browser-scan `{stamp}-competitors.json` sidecar (each item
+carries `.competitor` + `.platform`); HN / Stack Overflow / Bluesky come from per-alias API queries.
+- Content Volume: rough count of in-window mentions (High / Medium / Low or a number).
+- Sentiment: community stance TOWARD THAT COMPETITOR (🟢 favorable / ⚪ mixed / 🔴 unfavorable) — this is the
+  one place sentiment is about a competitor rather than our product.
+- Switching Signals: migration posts, scored from OUR product's perspective per the Directional rule
+  (FROM competitor → us = 🟢 win; FROM us → competitor = 🔴 loss; competitor↔competitor = ⚪).
+- Notable Items: 1–3 announcements / GA launches / outages / pricing changes, each with a validated link.
+This section lives INLINE in the one content report (never a separate file during a scan). The web UI's
+Reports → Competitors tab slices this section out and also lists any standalone on-demand `-competitors.md`
+deep-dive reports.
+-->
+
 ## Launch Coverage Tracker
 <!-- Include for: Product Marketer. Only generated when Events from config have dates in the scan window. -->
 <!-- Groups content by event, shows coverage angles and gaps. -->

.github/prompts/scout-scan.prompt.md

@@ -96,6 +96,7 @@ Run a content scan using the Content Scout agent.
      - If no fresh sidecars exist, attempt a refresh: run `node tools/browser-scan/index.mjs scan --slug {slug}` for each subject. If the command exits with a "no browser on CDP port" error, show the tip below **once** and continue — the API/RSS layers still cover X, LinkedIn, and Reddit with adequate fidelity.
      - One-time setup tip (show only when CDP is not running): *"For higher-fidelity X / LinkedIn / Reddit coverage, run `node tools/browser-scan/launch-edge.mjs` once, sign in to all three platforms, and leave the browser open. Future `/scout-scan` runs will use it automatically."*
      - When sidecars are available, ingest the freshest per platform and tag those items `*-browser` provenance — they take priority over Brave/RSS/old.reddit results, deduped by permalink. This includes the `*-content-sites.json` sidecar, which covers **Microsoft Tech Community, DZone, C# Corner, and Hashnode** (sources the API/RSS layers can't reach due to login wall / anti-bot / 500 / 404); its items carry `subSource` of `techcommunity` / `dzone` / `csharpcorner` / `hashnode` and feed the **content sections** (never Mindshare).
+     - When the config has a `## Competitors` section, the browser-scan also emits a `{stamp}-competitors.json` sidecar — Reddit + X items that name a tracked competitor, each tagged with the matched `competitor` and its `platform`. Feed these into the `## Competitor & Market Signals` section only (never the content sections or Mindshare). The `{stamp}-meta.json` sidecar's `competitors` block carries the per-competitor mention counts.
    - Search all enabled networks using the configured search terms. **Reddit, X/Twitter, LinkedIn, and Bluesky are always attempted** — never skipped just because one credential is missing (see "API Keys" exceptions in the agent definition for which layer/fallback to use when a key is empty). For Bluesky specifically: if both `BLUESKY_HANDLE` and `BLUESKY_APP_PASSWORD` are set in `.env`, you MUST call `createSession` and run the search; never write "credentials present but API call not completed."
    - **Open-web blog discovery is mandatory, not just RSS-by-tag.** The named blog platforms (Dev.to, Medium, Hashnode, …) miss self-hosted and vendor blogs. Run an **unrestricted** Brave web search (`q={term}` with no `site:` filter, `freshness=pm`) **once for every term in the config's `### Search Terms (text)` list** (not just one product term), drop hosts already covered by a dedicated layer (reddit/x/linkedin/youtube/github/stackoverflow/dev.to/medium/hashnode), and treat the rest as candidate blog posts to date-gate, relevancy-check, and score. Additionally, for any `## Known Author Watchlist` / `## Influencers to Monitor` entry that lists a blog domain (e.g., `benday.com/blog`), run a targeted `site:{domain}` Brave query per search term so that author's posts are caught. See "Blog Sources → Open-web blog discovery" in the agent definition.
    - Apply the content quality filter (date gate + relevancy gate). **Drop all hiring/recruiting/job-search content from EVERY section** (numbered tables, Conversations, Feature Requests, Influence Movers, social posts) per the "No Hiring Content" hard rule in the agent doc — even when the post mentions the product, even when the author is on the known-author list. Group these drops under a single `hiring/recruiting` counter in the JSON sidecar's `drop_reasons`; do not enumerate them in the user-facing summary.
@@ -110,6 +111,7 @@ Run a content scan using the Content Scout agent.
      - **Open-CFP gate (mandatory).** A conference belongs in `## Open Calls for Papers (CFPs)` **only if you have verified its CFP is still open** — i.e. it has a real submission page (Sessionize / Pretalx / typeform with a review process, per the CFP Vetting Checklist) AND a close date that is today or later. **Fetch the CFP page and confirm the close date before listing it** — a stale "CFP now open" banner does not count (e.g., a page may say "open" while its own milestones show the close date already passed). If you cannot verify an open close date ≥ today, the CFP does not go in the report. Never list a blog title, talk recap, or past event as a CFP.
      - When you confirm a close date during a scan, write it back to the config's `CFP Closes` column so future scans inherit it.
    - Populate the `## Mindshare` section with **date-gated** community social listening for THIS report's window only: include a post only if its publish date falls inside the report period. Never carry forward older posts to pad it. Conversation platforms only (Bluesky / X / LinkedIn / Reddit); exclude blogs, YouTube, GitHub, Stack Overflow, and official/owned handles. Do not add scan-date or rolling-window caveats to the body. The **X / LinkedIn / Reddit rows come primarily from the Layer 0 browser-scan sidecars** ingested in the browser-scan step above (`*-x.json` / `*-linkedin.json` / `*-reddit.json`); Bluesky rows come from the `searchPosts` API. Browser-scan `google-news` / `google-web` sidecar items are NOT social listening — they feed the content sections, never Mindshare.
+   - If **Competitor tracking** is enabled in config (`- **Competitor tracking:** on`) and the config's `## Competitors` section lists one or more products, populate the report's `## Competitor & Market Signals` section for those products (match on each competitor's name **and** its listed aliases). **Reddit + X competitor mentions come from the Layer 0 browser-scan `{stamp}-competitors.json` sidecar** produced in Step 0 (each item is tagged with the matched `competitor` and its `platform`) — ingest it first. Then cover the platforms browser-scan doesn't reach by running **per-alias API queries** on Hacker News (Algolia `search_by_date`), Stack Overflow, and Bluesky (`searchPosts`), **plus each competitor's own blog / release-notes / changelog** for the window. Fill one table row per competitor with — **Content Volume** (rough count of in-window mentions), **Sentiment** (community stance *toward that competitor*: 🟢 favorable / ⚪ mixed / 🔴 unfavorable), **Switching Signals** (migration posts to/from the competitor, scored from OUR product's perspective per the Directional rule: FROM competitor → us = 🟢 win, FROM us → competitor = 🔴 loss), and **Notable Items** (1–3 announcements, GA launches, outages, pricing changes) with **validated** links. Keep this **inline in the one content report** — do NOT write a separate `-competitors.md` file during a scan (see "One Scan = One Report"). A dedicated deep-dive competitor report is a separate, on-demand artifact. The web UI's **Competitors** tab surfaces this section automatically.
    - **Validate every URL before it goes in the report (mandatory).** Never present a link you have not confirmed is real, reachable content. Run `node tools/validate-urls.mjs <draft-or-url-list>` (or call the same logic via `tools/lib/url-validate.mjs`) over every external URL you intend to include. Drop or replace any link the check marks `DEAD` (HTTP 404/410, malformed, or a known-broken shape such as a LinkedIn `/feed/sdui-post/` permalink). Prefer the canonical source over an aggregator/redirect. If a worthwhile item has no navigable public link (common for LinkedIn SDUI posts), keep the item but state plainly that no public permalink exists rather than shipping a dead link. Login-walled platforms (x.com / linkedin.com / reddit.com / bsky.app / youtube.com) are exempt from the liveness probe but must still pass the shape check.
    - Number items sequentially across all sections.
 5. Save each topic's report to `reports/{YYYY-MM-DD-HHmm}-{slug}-content.md` (or `reports/{YYYY-MM-DD-HHmm}-content.md` if only one topic). **Exactly one report file per scan.** Every item from every layer (browser-scan sidecars, cascade fallbacks, RSS, APIs, MCP, manual imports) goes into that **one** file — never write a separate "supplemental", "addendum", or "sidecar report" alongside it. If a re-scan happens for the same window, edit the existing report file in place. See "One Scan = One Report" in the agent definition for the full rule.

CHANGELOG.md

@@ -4,6 +4,35 @@ All notable changes to Content Scout are tracked here.
 
 This project uses a product changelog version stream until formal release tags are cut. Minor feature releases use `0.x.0`; major fix bundles also receive their own `0.x.0` entry so every important fix has a durable version number.
 
+## [0.31.0] - 2026-07-02
+
+Competitor queries wired into the browser-scan conversations layer.
+
+### Versioned Features and Fixes
+
+| Version | Type | Area | Change |
+| --- | --- | --- | --- |
+| 0.31.0 | Minor feature | Browser scan / Competitor pass | New shared `tools/lib/competitors.mjs` parses the config's `## Competitors` section (bold name + `Aliases:`) into structured entries and provides query-term building + word-boundary matching/tagging. The browser-scan now runs a **competitor pass** over Reddit + X when competitors are configured: it queries the competitor names/aliases, keeps only items that name a tracked competitor, tags each with the matched `competitor`, and writes a `{stamp}-competitors.json` sidecar (plus a `competitors` block in `{stamp}-meta.json`). Because the browser-scan runs as Step 0 of every `/scout-scan`, the competitor pass happens **automatically alongside the monthly mindshare run** — no separate command. `/scout-scan` + agent docs ingest the sidecar into the `## Competitor & Market Signals` section and add Hacker News / Stack Overflow / Bluesky per-alias API coverage. Flags: `--no-competitors`, `--max-competitor-terms N` (default 12). |
+
+### Validation
+
+- New `tools/web-ui/test/competitors.test.js` (6 tests): parsing, alias handling, query-term building, word-boundary matching (no "Atlas" → "Atlassian" false positive), and item tagging.
+- `node --check` on the browser-scan orchestrator + a live `loadConfig` parse of the real config (8 competitors) confirm the wiring.
+
+## [0.30.0] - 2026-07-02
+
+Competitor sentiment & market-signal tracking.
+
+### Versioned Features and Fixes
+
+| Version | Type | Area | Change |
+| --- | --- | --- | --- |
+| 0.30.0 | Minor feature | Reports / Competitor signals | `/scout-scan` now populates a `## Competitor & Market Signals` section when **Competitor tracking** is on and the config's `## Competitors` section lists products: per-competitor content volume, community sentiment (toward the competitor), switching signals (migrations to/from, scored from our product's perspective), and notable announcements — matched by competitor name + aliases across conversation sources and the competitor's own blog/release-notes, inline in the single content report. The web UI **Reports** view gains a **Competitors** tab that lists standalone `-competitors.md` deep-dive reports and slices the Competitor & Market Signals section out of content reports (mirroring the Mindshare / CFPs & Events tabs). `tools/lib/doc-meta.mjs` classifies `-competitors.md` as kind "Competitors" and detects the section for tab filtering. |
+
+### Validation
+
+- Report classification + section detection verified against a generated `-competitors.md` report and a content report carrying the section.
+
 ## [0.29.0] - 2026-07-02
 
 Monthly roundup, content-originality review, responsive navigation, and scan-source hardening (`feature/roundup-techcommunity-scan`).

docs/WORKFLOW.md

@@ -143,6 +143,12 @@ Turn on **Originality scoring** in your config and each `/scout-scan` adds an `#
 
 It is a transparent review aid, **not** a verdict: a low score never drops or down-ranks an item, and titles/snippets return "insufficient text" rather than a guess. For a one-off check outside a scan, use `/scout-originality` (below).
 
+### Competitor Signals (optional)
+
+Turn on **Competitor tracking** and list rivals under `## Competitors` in your config (each a bold name plus optional `Aliases:`), and every `/scout-scan` adds a `## Competitor & Market Signals` section. The browser-scan Layer 0 runs a **competitor pass** over Reddit and X — writing a `{stamp}-competitors.json` sidecar with each mention tagged by the matched competitor — and the agent adds Hacker News, Stack Overflow, Bluesky, and each rival's own blog/release-notes. Each competitor gets a row scoring content volume, sentiment *toward that competitor*, switching signals (migrations to/from, from your product's perspective), and notable announcements.
+
+Because it runs inside the normal scan, a **monthly competitor pass happens automatically alongside your monthly mindshare** — there's no separate command. The web UI's **Reports → Competitors** tab surfaces the section (and any standalone deep-dive `-competitors.md` reports).
+
 ### Conversation Tracking
 
 Forums and social platforms are scanned separately from blog/article content. Conversations are tracked but not promoted as report items:

tools/browser-scan/README.md

@@ -220,6 +220,29 @@ Community needs you signed in (see setup); the other three need only a real
 browser. Hashnode posts on fully custom domains can't be pattern-matched
 here — the open-web Brave layer already covers those.
 
+### Competitor pass (one sidecar, tagged by competitor)
+
+When the loaded config has a `## Competitors` section, the scanner runs an
+extra **competitor pass** over Reddit and X after the main passes. It queries
+the configured competitor names + aliases (built by `competitorQueryTerms` in
+`tools/lib/competitors.mjs`), keeps only items that actually name a tracked
+competitor, and tags each with the matched competitor. Results merge into one
+`*-competitors.json` sidecar, shaped like the platform items above plus two
+fields:
+
+| Field | Meaning |
+|---|---|
+| `competitor` | The primary tracked competitor the item mentions (canonical name from config) |
+| `competitorMatches` | All tracked competitors the item mentions |
+
+The `{stamp}-meta.json` sidecar also gains a `competitors` block —
+`{ names, queryTerms, platforms, mentions, byCompetitor }` — for at-a-glance
+counts. The agent folds these into the report's **Competitor & Market
+Signals** section (never the content sections or Mindshare). Disable with
+`--no-competitors`; cap the query set with `--max-competitor-terms N`
+(default 12). Hacker News / Stack Overflow / Bluesky competitor coverage runs
+in the agent's API layer, not here.
+
 ## Rate-limit hygiene
 
 - One in-flight tab per platform; ≥3s between page loads.