Skip to content

Add find_image_equivalent to the AI Docs MCP server - #4101

Merged
matthewhelmke merged 1 commit into
matthewhelmke/docs-138-remove-image-toolsfrom
matthewhelmke/docs-138-find-image-equivalent
Sep 30, 2026
Merged

Add find_image_equivalent to the AI Docs MCP server#4101
matthewhelmke merged 1 commit into
matthewhelmke/docs-138-remove-image-toolsfrom
matthewhelmke/docs-138-find-image-equivalent

Conversation

@matthewhelmke

@matthewhelmke matthewhelmke commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

Adds find_image_equivalent, which answers "what Chainguard image replaces X?" from the Dockerfile Converter's mappings, and returns a reference the reader can actually pull. We tested it against a real AI tool. Before the change, 5 of 24 answers named the wrong image and 20 of 24 gave a cgr.dev/chainguard/... reference that anonymous pulls are refused. After it, 24 of 24 named the right image, and every answer gave a cgr.dev/<your-organization>/... reference.

Stacked on #4099. Merge order: #4099, then this PR, then #4100 (docs).

Why a tool, not search

I ran headless Claude Code against a local server, with only the AI Docs tools available, on 24 upstream images whose Chainguard name differs from the upstream one:

Search only (#4099) With find_image_equivalent
Correct image 19 / 24 24 / 24
Tool calls per question 1 to 4 1
Answers giving cgr.dev/chainguard/<image> 20 0
Answers giving cgr.dev/<your-organization>/<image> 0 24

Each of the five search misses was a confident wrong answer:

Separately, I probed all 21 cgr.dev/chainguard/<target> references the model printed, and the token endpoint refuses every one to anonymous callers.

Changes

  • find_image_equivalent(image) follows dfc's own matching order, from pkg/dfc/dfc.go:

    1. The reference with and without its tag.
    2. The last path segment alone.
    3. The name with any Docker Hub host removed, with and without library/.
    4. Prefix patterns such as nodejs*.

    It returns the image name, a cgr.dev/<your-organization>/... reference with any tag the mapping pins, the matched key, and a pointer to cg-oci for checking entitlement. With no match, it says what dfc would do (keep the image's own name) and points to the Directory.

  • generate_package_catalog.py writes dfc's 397 image mappings under a new image_mappings key. Reusing images would confuse them with the old catalog's image list.

  • The server instructions name the tool.

  • The unit-test job installs PyYAML, because the new tests read their ground truth from the YAML file.

Testing

  • pytest: 45 passed.
    • Ground truth is data/package-mappings.yaml, parsed separately from the code under test.
    • The tests cover every exact mapping (394 keys), tags, digests, Docker Hub hosts, prefix patterns, pinned tags, and registry ports.
    • The prefix, Docker Hub, and digest tests each fail when their rule is removed. The first digest test passed even with its rule removed, because a tag in front hid the bug, so it now also checks a digest with no tag.
  • End-to-end: I rebuilt the bundle and catalog from this branch and reran the same 24 Claude Code questions against a local server. Results are in the first table.

Refs DOCS-138

🤖 Generated with Claude Code

Created in collaboration with Claude Code running Opus 5.5 on 2026-09-30.

@matthewhelmke @claude
Add find_image_equivalent to the AI Docs MCP server
0137e7d 
A headless Claude Code test found that search usually surfaces dfc's
image mappings but hands them off unreliably: the matching line can
fall outside the returned snippet, and in one run the model misread a
correct snippet and invented details. A lookup returns one labeled
answer instead.

find_image_equivalent follows dfc's own matching order (pkg/dfc/dfc.go):
the reference with and without its tag, the last path segment, the
name with any Docker Hub host and library/ removed, then prefix
patterns such as nodejs*. It returns the Chainguard image name and a
pull reference of the form cgr.dev/<your-organization>/<image>, and
never presents the public cgr.dev/chainguard namespace as the place to
pull it.

generate_package_catalog.py now writes dfc's image mappings to
package-mappings.json under image_mappings, a new key, so it cannot be
mistaken for the image list the old catalog carried under images.

Tests take their ground truth from data/package-mappings.yaml, parsed
separately, and cover every exact mapping plus tags, digests, Docker
Hub hosts, prefix patterns, pinned tags, and registry ports. The prefix,
Docker Hub, and digest tests each fail with their rule removed. The
unit-test job now installs PyYAML for them.

Refs DOCS-138

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 30, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
chainguard-docs-preview Ready Ready Preview Sep 30, 2026 2:07pm UTC

Request Review

@matthewhelmke
matthewhelmke requested review from a team as code owners September 30, 2026 14:06
This was referenced Sep 30, 2026
Merged
Merged
@matthewhelmke
matthewhelmke merged commit 0137e7d into matthewhelmke/docs-138-remove-image-tools Sep 30, 2026
8 checks passed
@matthewhelmke
matthewhelmke deleted the matthewhelmke/docs-138-find-image-equivalent branch September 30, 2026 14:13
@matthewhelmke

matthewhelmke commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator Author

This change moved into #4099, so this PR shows as merged into #4099's branch rather than into main. Nothing reached main from here. Its one commit, 0137e7dcd, now ships as part of #4099.

Why: this PR was stacked on #4099. On its own, #4099's search-only instructions still led some AI tools to guess image names, and find_image_equivalent is what fixed that. Folding the commit in means one review, keeps the two from merging separately, and avoids rebasing this branch after #4099 squash-merges. The test results and rationale from this description are now in #4099's.

matthewhelmke added a commit that referenced this pull request Sep 30, 2026
@matthewhelmke @claude
Remove the AI Docs MCP server's frozen image tools; restore dfc mappi…
286d599 
…ngs (#4099)

The AI Docs MCP server's image tools served year-old data. ServiceNow's
AI tool got back image references that don't exist, such as
`cgr.dev/chainguard/victoriametrics` (ZD 11898), and that still
reproduced on the live server today. This PR removes those tools and
tells connecting clients to use `cg-oci` instead, which reads the
registry live and was documented in #4053. It also adds
`find_image_equivalent`, which answers "what Chainguard image replaces
X?" from the Dockerfile Converter mappings with a reference the reader
can actually pull. In a test with a real AI tool, correct answers went
from 19 of 24 to 24 of 24. Unpullable `cgr.dev/chainguard/...`
references went from 20 of 24 answers to none.

## Changes

**Remove the image tools and frozen sources** (DOCS-138, DOCS-139,
DOCS-140)

- Removes `get_image_docs`, `list_images`, and `check_image_freshness`,
along with the registry client and image parsing behind them. All three
depended on an `images-private` README snapshot that was uploaded by
hand on 2025-08-12. The automated export never succeeded.
- Adds MCP `instructions`, sent at `initialize`, that point clients to
`cg-oci`, `cg-apk`, and `cg-versions`.
- Stops compiling image READMEs and courses into the bundle. Courses
already contributed no content, because the 2025-08-12 tarball didn't
match the compiler's expected layout.
- Drops the workflow's GCS download step. The only other thing it
downloaded, the edu export, was extracted and never read. GCP auth stays
for Artifact Registry, Cloud Run, and the bundle upload.
- Renames `generate_image_catalog.py` to `generate_package_catalog.py`
and ships its output as `package-mappings.json`. The build still ships
`image-catalog.json` with the same content, so existing `CATALOG_PATH`
settings keep working.
- The bundle drops from about 14.9 MB to 4.5 MB.

**Compile the dfc mappings into the bundle again** (fixes DOCS-174)

- The compiler reads the committed `data/package-mappings.yaml`, which
the nightly `autodocs-platform` job copies from dfc. The unmaintained
GCS dfc object is no longer used.
- The section uses the compiler's page signature, so search results link
to the Package and image name mappings page. A missing file now prints a
warning instead of passing silently.

**Search: match "image" in queries to "Container" in titles**

- With the image READMEs gone, "python image" stopped finding "Getting
started with the Python Chainguard Container." A title-only synonym
brings it back.
- Known limit: a page whose body contains the literal phrase, such as
"nginx image," still collects the existing exact-phrase bonus and can
outrank the matching guide.

**Add `find_image_equivalent`, and point clients at the pullable
reference form**

This came out of review testing and was folded in from #4101. I ran
headless Claude Code against a local server, with only the AI Docs tools
available, on 24 upstream images whose Chainguard name differs from the
upstream one:

| | Search only | With `find_image_equivalent` |
| --- | --- | --- |
| Correct image | 19 / 24 | **24 / 24** |
| Tool calls per question | 1 to 4 | 1 |
| Answers giving `cgr.dev/chainguard/<image>` | 20 | **0** |
| Answers giving `cgr.dev/<your-organization>/<image>` | 0 | **24** |

Each of the five search misses was a confident wrong answer:

- In two runs, the model skipped the server because the first
instructions said it had "no container image data."
- In two runs, the matching line fell outside the returned snippet.
- In one run, the model misread a correct snippet and invented details.

All 21 `cgr.dev/chainguard/<target>` references the model printed are
refused to anonymous callers at the token endpoint.

- **`find_image_equivalent(image)`** follows dfc's own matching order,
from `pkg/dfc/dfc.go`:
  1. The reference with and without its tag.
  2. The last path segment alone.
3. The name with any Docker Hub host removed, with and without
`library/`.
  4. Prefix patterns such as `nodejs*`.

It returns the image name, a `cgr.dev/<your-organization>/...` reference
with any tag the mapping pins, the matched key, and a pointer to
`cg-oci` for checking entitlement.
- **`generate_package_catalog.py`** writes dfc's 397 image mappings
under a new `image_mappings` key, so they can't be confused with the old
catalog's image list.
- **The server instructions and the bundle's DFC section** say the
mappings are available, name the tool, and give the reference form
`cgr.dev/<your-organization>/<image>`.
- **The `unit-test` job** installs PyYAML, because the new tests read
their ground truth from the YAML file.

**Also**

- `compile_docs.py` gets a formatting-only `black` commit first; its AST
is unchanged. Without it, the pre-commit hook would reformat the whole
file inside the real change.
- `quick-start-guide.md` gets its existing lint errors fixed. It now
passes through the same cleaning step as every page, so its markdownlint
directive stays out of the bundle.
- Fixes the image reference in `extract.sh`'s usage line
(`cgr.dev/chainguard/ai-docs` doesn't exist).

The docs pages follow in #4100, merged right after this one.

## Testing

- `pytest scripts/test_mcp_server.py scripts/test_mcp_transport.py
scripts/test_compile_docs.py`: 45 passed.
- The new tests check the exact tool set, the instructions, the DFC
section against the committed YAML, the synonym, and
`find_image_equivalent`.
- The image lookup tests take ground truth from
`data/package-mappings.yaml`, parsed separately. They cover all 394
exact mappings, plus tags, digests, Docker Hub hosts, prefix patterns,
pinned tags, and registry ports.
- The synonym, DFC, prefix, Docker Hub, and digest tests each fail when
their change is reverted.
- Compiled a real bundle and catalog locally: 4.47 MB, 626 pages, no
image markers, 683 package mappings, and both catalog filenames
byte-identical.
- Ran the server over HTTP against those files:
- `tools/list` returns the documentation tools, and `initialize` carries
the instructions. This run predates the `find_image_equivalent` commit,
which the 24-run test and CI's `smoke-test` and `unit-test` jobs cover.
  - `find_package_equivalent("build-essential")` returns `build-base`.
  - A dfc search returns the new section and its page URL.
  - Calling `list_images` returns `Unknown tool`.
- Built the container with the real artifacts. `smoke-test-mcp.sh`
passes, `verify` passes, and `extract` writes both catalog files with
valid checksums.

## After merge

The `scripts/**` push trigger redeploys the hosted server. The served
download at `/downloads/chainguard-complete-docs.md` shrinks at the next
nightly publish. Until then, the real-bundle tests run against the old
committed bundle, which still passes.

Refs DOCS-138, DOCS-139, DOCS-140
Fixes DOCS-174

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---
Created in collaboration with Claude Code running Opus 5.5 on
2026-09-30.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
matthewhelmke added a commit that referenced this pull request Sep 30, 2026
@matthewhelmke @claude
Merge Developer Resources into the AI Docs page and document the imag…
e79b8d9 
…e-tool replacements (#4100)

The AI Docs page now documents the four tools the server keeps. It also
includes a table that sends anyone who relied on the removed image tools
to the live replacements from #4053. Developer Resources, which covered
the same product and had drifted, is merged in.

**Merge after #4099.** This page describes the server as #4099 leaves
it, including the `find_image_equivalent` tool, which was folded into
#4099 from #4101.

## Changes

**`/platform/mcp-servers/ai-docs/`**, retitled "AI Docs: Chainguard
documentation for AI tools"

- Documents five tools, including `find_image_equivalent`.
- A new "Image references" section explains that most Chainguard images
are pulled from `cgr.dev/<your-organization>/<image>`, and that the
public `cgr.dev/chainguard` namespace serves only a free subset.
- A new "Container image data" section has a table mapping each task to
its former AI Docs tool and its replacement:
- `get_image_docs`: the Chainguard Containers Directory, or `get_config`
and `get_apko_config` on `cg-oci`
  - `list_images`: `list_repos` on `cg-oci`
- `check_image_freshness`: `list_tags` and `get_manifest` on `cg-oci`,
or `registry_tags_list` on `cg-api`
- `find_package_equivalent`: unchanged, with `cg-apk` for package
versions
- A dated note records the removal. We can delete the note and the
"Former AI Docs tool" column once migrators have moved on.
- "Image catalog" becomes "Package catalog" and covers
`package-mappings.json` and its earlier name, `image-catalog.json`. The
standalone and Claude Desktop examples use the new name.
- The `check_image_freshness` example becomes a dfc image-mapping
example.
- A new "Use the documentation without a server" section holds the
offline content from Developer Resources: a direct link to the download,
which that page never had, plus container `verify` and `extract`,
`cosign verify`, and how to load the bundle into an AI tool. It notes
that the bundle is larger than most tools can hold in one conversation.
- Adds a troubleshooting entry for an AI tool that can't find image
details.

**`/developer-resources/`**
- Deleted and turned into an alias of the AI Docs page. The page
repeated the AI Docs tool list and Claude Desktop configuration, and its
copies had drifted: it still claimed Alpine support.
- Its roughly 50 example prompts are dropped. The per-tool prompts and
the bundle's own usage guide cover the same ground.
- The sidebar's **AI Docs Bundle** link now points to
`#use-the-documentation-without-a-server`.

**Other pages**
- **MCP servers overview:** no longer says AI Docs searches image
READMEs, and points to `cg-oci` for image data.
- **AI documentation security:** lists the two remaining sources (edu
pages and the dfc mappings), drops the provisional dfc note, and says a
clone is all the compiler needs.
- **Directory name:** the AI Docs page now uses "Chainguard Containers
Directory" instead of "Images Directory."

## Testing

- `hugo --gc --minify` builds clean. `/developer-resources/` redirects
to `/platform/mcp-servers/ai-docs/`.
- Every link and fragment in the article body of the three changed pages
resolves against the build, including the sidebar link.
- The migration table and note render correctly; there's no literal
shortcode text.
- `check-related-pages` found no page that needs this change. The Kiro
and Cursor pages still say "Chainguard Images directory." That belongs
to a sitewide naming sweep, so this PR leaves them alone.

Refs DOCS-138, DOCS-174

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---
Created in collaboration with Claude Code running Opus 5.5 on
2026-09-30.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 0137e7dc Deployed Sep 30, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@matthewhelmke