Support GitHub App installation tokens for the GitHub forge#6806
Support GitHub App installation tokens for the GitHub forge#6806slayerjain wants to merge 5 commits into
Conversation
When WOODPECKER_GITHUB_APP_ID and WOODPECKER_GITHUB_APP_PRIVATE_KEY(_FILE) are configured, the GitHub forge authenticates repo-scoped server-side operations (commit statuses, pipeline config fetching, changed-file lookups, branch/PR queries) and clone credentials with GitHub App installation access tokens instead of the repo owner's OAuth token, falling back to user OAuth tokens for repositories the app is not installed on. This raises API rate limits for large organizations (per-installation limits that scale with repository count instead of a single user's 5,000 req/h), keeps pipelines working when the enabling user loses access, and posts commit statuses under the app's bot identity. Includes: app JWT signing (client id or app id as issuer), cached installation lookups and tokens with validity margins, write-only private key handling in the forge API/UI, save-time and boot-time validation of app credentials, an admin app-health check endpoint with a UI test button, admin form fields, docs, and tests. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
|
Surge PR preview deployment succeeded. View it at https://woodpecker-ci-woodpecker-pr-6806.surge.sh |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6806 +/- ##
==========================================
+ Coverage 47.30% 48.31% +1.00%
==========================================
Files 440 441 +1
Lines 29726 30007 +281
==========================================
+ Hits 14062 14498 +436
+ Misses 14496 14313 -183
- Partials 1168 1196 +28 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Adds tests for the forge API handlers (redaction, secret merge, save-time validation, app health), the installation token error and cache-invalidation branches, PEM repair edge cases, repo-scoped forge methods using installation tokens, RedactSecrets, setupGitHub option coercion, and the env boot path validation. Also removes an unreachable nil-map guard in restoreSecretOptions. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
|
This seems to be working fine in Keploy on-prem CI. Please let me know if any changes needed before merge. |
|
|
||
| ## GitHub App | ||
|
|
||
| In addition to the OAuth login, Woodpecker can authenticate as a [GitHub App](https://docs.github.com/en/apps/creating-github-apps/about-creating-github-apps/about-creating-github-apps) using installation access tokens for server-side API calls (commit statuses, pipeline configuration fetching, changed-file lookups) and for cloning repositories. |
There was a problem hiding this comment.
... for cloning repos
this could have security implications and need to be looked at
There was a problem hiding this comment.
Makes sense. I have narrowed the minted token to just the job's repo and with read-only content permissions.
There was a problem hiding this comment.
Please resolve this if you think it's fine.
- scope clone credentials to the repository being built with read-only contents access by default; WOODPECKER_GITHUB_APP_CLONE_TOKEN_SCOPE (or the admin UI) can widen them to the whole app installation for private git submodules and similar cross-repository clones - centralize the additional-option keys as constants and make the secret option redaction and restore logic generic per forge type - validate all forge types (except addons) when saving via the API and reject non-string github app options instead of coercing them - relax the bitbucket datacenter admin-scope option to an optional boolean, it is not exposed by the admin UI - group the forge id routes in the router - document the clone token scope and reference woodpecker-ci#2851 for the queued token expiry limitation Signed-off-by: slayerjain <shubhamkjain@outlook.com>
This adds first-class GitHub App support to the GitHub forge: when
WOODPECKER_GITHUB_APP_IDandWOODPECKER_GITHUB_APP_PRIVATE_KEY(_FILE) are configured, Woodpecker authenticates repo-scoped server-side operations and clone credentials with installation access tokens instead of the repo owner's personal OAuth token. User login stays on the OAuth flow, and repositories the app is not installed on transparently fall back to the current user-token behavior.Why
Today every server-side API call (commit statuses, pipeline config fetching, changed-file lookups) and every clone runs on the OAuth token of the user who enabled the repository. For larger organizations this has three structural problems:
Installation tokens get per-installation rate limits that scale with repository count, survive user departure, and post statuses as the app's bot account. This also allows removing the long-stale docs warning against GitHub Apps (added in #2499, obsolete since #3811 introduced refresh-token support).
What changed
server/forge/github/app.go: authenticates as the app (RS256 JWT via the already-vendoredgolang-jwt/jwt/v5; client id or app id as issuer, backdatediatper GitHub's guidance) and mints installation access tokens through the existinggo-githubdependency — no new module needed. Installation lookups and tokens are cached with validity margins (5 min for API calls, 45 min for clone credentials); "not installed" results are negative-cached and uninstall/suspension invalidates the cache and falls back gracefully.Netrc(x-access-token),Status,File/Dir,Branches,BranchHead,PullRequestsand the webhook changed-files/tag loaders. The loaders skip the repo-owner DB lookup entirely when the app covers the repo. Tokens are installation-wide on purpose so same-org submodule clones keep working.AdditionalOptions["app-id"/"app-private-key"](same channel as the bitbucket-dc machine account) →github.Opts. No DB migration needed. Private keys are accepted as PEM or base64-encoded PEM, including keys whose newlines were mangled by env files or input fields.oauth_client_secret): responses replace it with anapp-private-key-setmarker, and updates that omit it keep the stored key. Clearing the app id disables app auth again.GET /forges/{id}/app-healthplus a "Test GitHub App" button in the admin forge form so operators can verify the credentials actually work.Notes
ghinstallation-based tools (Atlantis, ArgoCD).