Skip to content

feat(cloudflare): mirror method-page specs, drop the in-repo snapshot - #534

Open
Mkassabov wants to merge 1 commit into
mainfrom
fix/cloudflare-spec-mirror
Open

feat(cloudflare): mirror method-page specs, drop the in-repo snapshot#534
Mkassabov wants to merge 1 commit into
mainfrom
fix/cloudflare-spec-mirror

Conversation

@Mkassabov

@Mkassabov Mkassabov commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Warning

Do not merge this PR by itself. Always merge the entire stack — click Merge on the top PR (#536). We only split this because GitHub's Files tab caps at 3,000 files and the spec-snapshot deletion ate the UI. Merging any layer alone leaves main inconsistent (fetcher without regen, Smithy without TS, or vice versa).

Stack (bottom → top): #534 fetcher#535 generated specs#536 services

This layer

Unblocks spec-mirror-cloudflare now that method-page index.md URLs on developers.cloudflare.com serve markdown again.

  • spec-repos/cloudflare/fetch-specs.ts crawls the /api/ sidebar and downloads only /methods/ pages. Resource index pages still return multi-MB HTML and are skipped.
  • About 10% of method pages (Workers, DNS, Zero Trust, …) still return HTML; those are converted from stldocs-* markup into the pre-Starlight markdown dialect.
  • spec-to-smithy.ts parses both today’s Starlight dump and that legacy dialect, and reads specs/spec-mirror-cloudflare/specs/api/resources via resolveSpecPath.
  • The committed packages/cloudflare/specs/api snapshot is gone (~3.3M lines). .gitmodules points at distilled-mirror/spec-mirror-cloudflare (shallow = true).
  • pnpm specs:check is green (27 mirrors).

Until this stack merges and the distilled-submodules stack deploys the fetch script into the (currently empty) mirror, regenerate with:

pnpm specs:local cloudflare
DISTILLED_SPECS_LOCAL=1 pnpm generate cloudflare

Unblock spec-mirror-cloudflare now that method-page index.md URLs serve
markdown again. Fetch only /methods/ pages (HTML resource indexes skipped;
heavy method pages converted from stldocs markup), point spec-to-smithy at
the mirror, and remove the committed packages/cloudflare/specs/api crawl.
@Mkassabov
Mkassabov force-pushed the fix/cloudflare-spec-mirror branch from 54df6c7 to 0ff2893 Compare August 31, 2026 08:47
@Mkassabov Mkassabov changed the title feat(cloudflare): mirror method-page specs and regenerate the SDK feat(cloudflare): mirror method-page specs, drop the in-repo snapshot Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant