Skip to content

fix: download schema via REST API to avoid raw.githubusercontent.com 429s - #264

Draft
stekern wants to merge 2 commits into
mainfrom
fix/gp-config-schema-download-429
Draft

fix: download schema via REST API to avoid raw.githubusercontent.com 429s#264
stekern wants to merge 2 commits into
mainfrom
fix/gp-config-schema-download-429

Conversation

@stekern

@stekern stekern commented Jul 6, 2026

Copy link
Copy Markdown
Member

Problem

build-gp-config lets check-jsonschema download the schema referenced by the config's $schema property directly from raw.githubusercontent.com. Unauthenticated requests to that host are rate-limited per IP, and GitHub-hosted runners share egress IPs with other tenants, so validation fails intermittently with HTTP 429 regardless of our own usage:

Error: Unexpected Error building schema validator
FailedDownloadError: got response with status=429, retries exhausted

Sending an auth token does not help — raw.githubusercontent.com ignores tokens for rate-limiting purposes (see this community discussion).

Fix

  • When $schema is a raw.githubusercontent.com URL, download the schema through the REST API contents endpoint instead (Accept: application/vnd.github.raw), where the workflow token gets its own documented per-repository rate limit, then validate against the local copy. Local file paths (used by the CI tests here) pass through unchanged.
  • Handles both plain (OWNER/REPO/REF/PATH) and refs/heads/- / refs/tags/-prefixed raw URL forms; unparseable URLs fail with a clear error.
  • Replace pip install check-jsonschema with uvx, isolated from the runner's system Python (the action already installs uv). The Renovate pin comment still matches the custom regex manager.

Notes

  • Known limitation: branch refs containing slashes (e.g. refs/heads/feature/foo) are inherently ambiguous in raw URLs and are not supported; the failure is loud, and schema URLs use single-segment tags by convention.
  • Validating against a downloaded copy means relative external $refs would no longer resolve; the gp-cicd schema only uses internal #/$defs/ refs today.

stekern added 2 commits July 6, 2026 15:54
…429s

Unauthenticated fetches from raw.githubusercontent.com are rate-limited
per IP, and GitHub-hosted runners share egress IPs, so schema validation
failed intermittently with HTTP 429. Download the schema through the
REST API instead, where the workflow token has its own per-repository
rate limit. Also replace pip install with uvx.
Use = instead of the bash-only == in tests, and a case statement
instead of [[ ]] glob matching for the URL prefix check.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant