-
Notifications
You must be signed in to change notification settings - Fork 0
Add the Socket Basics scanner infrastructure #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| name: Fetch and merge Socket suppression config | ||
| description: > | ||
| Fetches the org-level and repo-level Socket Basics suppression configs from the private | ||
| ynab-sast-scanner-suppressions repo and merges them into a single config file for | ||
| socket-basics --config. | ||
|
|
||
| inputs: | ||
| suppressions-token: | ||
| description: > | ||
| GitHub App installation token with contents:read on the suppressions repo. | ||
| required: true | ||
| suppressions-repo: | ||
| description: Repository holding the suppression configs. | ||
| required: false | ||
| default: ynab/ynab-sast-scanner-suppressions | ||
| suppressions-ref: | ||
| description: > | ||
| Ref to read the configs from. Defaults to the default branch, so a suppression PR | ||
| takes effect on merge without a second PR here. Pin to a SHA if you would rather | ||
| changes be deliberate. | ||
| required: false | ||
| default: "" | ||
| target-repo: | ||
| description: 'Repo being scanned, as owner/name. Selects suppressions/<name>.json.' | ||
| required: true | ||
|
|
||
| outputs: | ||
| config-path: | ||
| description: Absolute path to the merged config file. | ||
| value: ${{ steps.merge.outputs.config-path }} | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - id: fetch | ||
| shell: bash | ||
| env: | ||
| GH_TOKEN: ${{ inputs.suppressions-token }} | ||
| SUPPRESSIONS_REPO: ${{ inputs.suppressions-repo }} | ||
| SUPPRESSIONS_REF: ${{ inputs.suppressions-ref }} | ||
| TARGET_REPO: ${{ inputs.target-repo }} | ||
| run: | | ||
| set -euo pipefail | ||
| dir="${RUNNER_TEMP}/socket-suppressions" | ||
| mkdir -p "$dir" | ||
|
|
||
| ref_qs="" | ||
| if [[ -n "$SUPPRESSIONS_REF" ]]; then | ||
| ref_qs="?ref=${SUPPRESSIONS_REF}" | ||
| fi | ||
|
|
||
| # Fetch the two files by API rather than checking the repo out. actions/checkout | ||
| # can only write inside the workspace, and the workspace is the scan target — a | ||
| # private config checked out there would land in the scanned tree. | ||
| if ! gh api "repos/${SUPPRESSIONS_REPO}/contents/org.json${ref_qs}" \ | ||
| --jq '.content' | base64 -d > "$dir/org.json"; then | ||
| echo "::error::Could not read org.json from ${SUPPRESSIONS_REPO}. Check that the" \ | ||
| "GitHub App is installed on that repo with contents:read, and that its" \ | ||
| "installation token reached this workflow." | ||
| exit 1 | ||
| fi | ||
|
|
||
| repo_name="${TARGET_REPO##*/}" | ||
| if gh api "repos/${SUPPRESSIONS_REPO}/contents/suppressions/${repo_name}.json${ref_qs}" \ | ||
| --jq '.content' 2>/dev/null | base64 -d > "$dir/repo.json"; then | ||
| echo "Repo-level suppressions found for ${repo_name}." | ||
| else | ||
| rm -f "$dir/repo.json" | ||
| echo "No repo-level suppressions for ${repo_name}; using org config only." | ||
| fi | ||
|
|
||
| - id: merge | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| dir="${RUNNER_TEMP}/socket-suppressions" | ||
| out="${dir}/merged.json" | ||
| python3 "${{ github.action_path }}/merge_config.py" \ | ||
| "$dir/org.json" "$dir/repo.json" "$out" | ||
| echo "config-path=$out" >> "$GITHUB_OUTPUT" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| #!/usr/bin/env python3 | ||
| """ | ||
| Merges the org-level and repo-level Socket Basics suppression configs into one file | ||
| for socket-basics --config. | ||
|
|
||
| Usage: | ||
| merge_config.py <org.json> <repo.json|missing path> <output.json> | ||
|
|
||
| The repo path is allowed not to exist: most repos have no repo-level suppressions. | ||
| """ | ||
| import json | ||
| import re | ||
| import sys | ||
|
|
||
| # Anchored at line start deliberately: a `//` inside a string (a URL, say) must survive. | ||
| _LINE_COMMENT = re.compile(r"^\s*//.*$", re.MULTILINE) | ||
|
|
||
|
|
||
| def load(path): | ||
| with open(path) as f: | ||
| return json.loads(_LINE_COMMENT.sub("", f.read()) or "{}") | ||
|
|
||
|
|
||
| def merge(org_path, repo_path, merged_path): | ||
| org = load(org_path) | ||
|
|
||
| try: | ||
| repo = load(repo_path) | ||
| except FileNotFoundError: | ||
| repo = {} | ||
|
|
||
| # `_meta` carries suppression justifications for humans and CODEOWNERS review. | ||
| org.pop("_meta", None) | ||
| repo.pop("_meta", None) | ||
|
|
||
| merged = {**org, **repo} | ||
|
|
||
| # Repo-level rule lists are additive: a repo file names only the rules it needs on | ||
| # top of the org list, never a copy of it. Every other key is a plain override. | ||
| for key in merged: | ||
| if key.endswith("_disabled_rules"): | ||
| org_val = org.get(key, "") | ||
| repo_val = repo.get(key, "") | ||
| parts = [r.strip() for r in f"{org_val},{repo_val}".split(",") if r.strip()] | ||
| seen: set = set() | ||
| merged[key] = ",".join(r for r in parts if not (r in seen or seen.add(r))) | ||
|
|
||
| with open(merged_path, "w") as f: | ||
| json.dump(merged, f, indent=2) | ||
|
|
||
| # Counts only — enough to debug a merge without disclosing which rules are off. | ||
| summary = ", ".join( | ||
| f"{k.removesuffix('_disabled_rules')}={len(v.split(','))}" | ||
| for k, v in sorted(merged.items()) | ||
| if k.endswith("_disabled_rules") and v | ||
| ) | ||
| print(f"Merged Socket config written to {merged_path}") | ||
| print(f"Suppressed rule counts by language: {summary or '(none)'}") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| if len(sys.argv) != 4: | ||
| sys.exit(f"usage: {sys.argv[0]} <org.json> <repo.json> <output.json>") | ||
| merge(sys.argv[1], sys.argv[2], sys.argv[3]) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| name: Socket Basics Security Scan | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| suppressions-ref: | ||
| description: Ref to read suppression configs from. Defaults to their default branch. | ||
| required: false | ||
| type: string | ||
| default: "" | ||
| secrets: | ||
| SOCKET_SECURITY_API_KEY: | ||
| required: true | ||
| SAST_SUPPRESSIONS_APP_PRIVATE_KEY: | ||
| description: > | ||
| Private key for the GitHub App that can read ynab-sast-scanner-suppressions. | ||
| The App id is not a secret and is set below. | ||
| required: true | ||
|
|
||
| concurrency: | ||
| group: socket-basics-${{ github.repository }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| env: | ||
| # App ids are not secrets. Only the private key is. | ||
| SAST_SUPPRESSIONS_APP_ID: "4706264" | ||
|
|
||
| jobs: | ||
| security-scan: | ||
| permissions: | ||
| issues: write | ||
| contents: read | ||
| pull-requests: write | ||
| runs-on: blacksmith-2vcpu-ubuntu-2404 | ||
| timeout-minutes: 15 | ||
| steps: | ||
| - name: Reject non-pull_request callers | ||
| if: github.event_name != 'pull_request' | ||
| env: | ||
| CALLER_EVENT: ${{ github.event_name }} | ||
| CALLER_REPO: ${{ github.repository }} | ||
| run: | | ||
| echo "::error::Socket Basics was called on '${CALLER_EVENT}' by ${CALLER_REPO}, but this reusable workflow only supports 'pull_request'." | ||
| echo "::error::Its concurrency group cancels superseded runs, which on a non-pull_request event can discard the scan record for a commit that shipped." | ||
| echo "::error::To support '${CALLER_EVENT}', scope the concurrency group by event or drop cancel-in-progress in ynab/ynab-sast-scanner/.github/workflows/socket-basics.yml, then relax this check." | ||
| exit 1 | ||
|
|
||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
|
|
||
| # Mints a short-lived installation token scoped to the one private repo holding our | ||
| # suppression configs. Deliberately an App rather than a PAT: it expires on its own, | ||
| # is scoped to a single repo, and has no human owner to offboard. | ||
| - name: Mint a token for the suppressions repo | ||
| id: suppressions-token | ||
| uses: actions/create-github-app-token@a8d616148505b5069dccd32f177bb87d7f39123b # v2.1.4 | ||
| with: | ||
| app-id: ${{ env.SAST_SUPPRESSIONS_APP_ID }} | ||
| private-key: ${{ secrets.SAST_SUPPRESSIONS_APP_PRIVATE_KEY }} | ||
| owner: ${{ github.repository_owner }} | ||
| repositories: ynab-sast-scanner-suppressions | ||
|
sgrammargs marked this conversation as resolved.
|
||
|
|
||
| - name: Fetch and merge suppression config | ||
| id: config | ||
| uses: ynab/ynab-sast-scanner/.github/actions/merge-socket-config@04fdc1c6dcf163ce3639f879b95487bc064c0210 | ||
| with: | ||
| suppressions-token: ${{ steps.suppressions-token.outputs.token }} | ||
| suppressions-ref: ${{ inputs.suppressions-ref }} | ||
| target-repo: ${{ github.repository }} | ||
|
|
||
| - name: Run Socket Basics | ||
| env: | ||
| # Bump this digest when upgrading socket-basics. Currently v3.0.0 - https://github.com/SocketDev/socket-basics/pkgs/container/socket-basics. | ||
| SOCKET_BASICS_IMAGE: ghcr.io/socketdev/socket-basics@sha256:93ce10202376b57ec5ed428d55337d4edd62ae2653434ac38301b55f6d230975 | ||
| GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} | ||
| SOCKET_SECURITY_API_KEY: ${{ secrets.SOCKET_SECURITY_API_KEY }} | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
| MERGED_CONFIG: ${{ steps.config.outputs.config-path }} | ||
| run: | | ||
| # A leftover facts file must never masquerade as this scan's results. | ||
| rm -f "${GITHUB_WORKSPACE}/.socket.facts.json" | ||
|
|
||
| # The merged config lives in RUNNER_TEMP, which is mounted below, so the | ||
| # suppression list never enters the scanned workspace. | ||
| config_in_container="/github/runner_temp/${MERGED_CONFIG#$RUNNER_TEMP/}" | ||
|
|
||
| docker run --rm \ | ||
| -e INPUT_SOCKET_ORG="ynab" \ | ||
| -e INPUT_SOCKET_SECURITY_API_KEY="$SOCKET_SECURITY_API_KEY" \ | ||
| -e GITHUB_TOKEN \ | ||
| -e GITHUB_PR_NUMBER \ | ||
| -e GITHUB_REPOSITORY \ | ||
| -e GITHUB_REPOSITORY_OWNER \ | ||
| -e GITHUB_SHA \ | ||
| -e GITHUB_REF \ | ||
| -e GITHUB_REF_NAME \ | ||
| -e GITHUB_HEAD_REF \ | ||
| -e GITHUB_BASE_REF \ | ||
| -e GITHUB_EVENT_NAME \ | ||
| -e GITHUB_WORKSPACE=/github/workspace \ | ||
| -e GITHUB_RUN_ID \ | ||
| -e GITHUB_RUN_NUMBER \ | ||
| -e GITHUB_ACTIONS \ | ||
| -e CI \ | ||
| -v "${GITHUB_WORKSPACE}:/github/workspace" \ | ||
| -v "${RUNNER_TEMP}:/github/runner_temp" \ | ||
| -v "${RUNNER_TEMP}/_github_home:/github/home" \ | ||
| -v "${RUNNER_TEMP}/_github_workflow:/github/workflow" \ | ||
| -v "${RUNNER_TEMP}/_runner_file_commands:/github/file_commands" \ | ||
| -w /github/workspace \ | ||
| "$SOCKET_BASICS_IMAGE" \ | ||
| --config "$config_in_container" | ||
|
|
||
| - name: Verify the scan produced results | ||
| # The scan step's exit code covers findings and crashes, but not a clean exit that | ||
| # wrote nothing — an empty scan would otherwise read as a pass. This inherits the | ||
| # default success() condition on purpose: a non-zero scan step has already failed | ||
| # the job, and a cancelled run (see the concurrency group above) has no results to | ||
| # check. | ||
| run: | | ||
| if [[ ! -f "${GITHUB_WORKSPACE}/.socket.facts.json" ]]; then | ||
| echo "::error::Socket Basics exited successfully but wrote no .socket.facts.json — failing the job instead of reporting a false pass." | ||
| exit 1 | ||
| fi | ||
| echo "Socket Basics wrote .socket.facts.json ($(wc -c < "${GITHUB_WORKSPACE}/.socket.facts.json") bytes)." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # Scan artifacts. These contain code snippets from whatever was scanned, and this | ||
| # repo is public — they must never be committed. | ||
| .socket-scans/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # This is the scanning mechanism for Socket Basics (SAST). | ||
| * @ynab/security |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,76 @@ | ||
| # ynab-sast-scanner | ||
| Shared GH workflow used for SAST scanning against YNAB code bases. | ||
|
|
||
| The reusable GitHub Actions workflow that runs [Socket Basics](https://github.com/SocketDev/socket-basics) | ||
| static analysis on every pull request across YNAB repositories. | ||
|
|
||
| ## Using it | ||
|
|
||
| Add this to a repository as `.github/workflows/socket-basics.yml`: | ||
|
|
||
| ```yaml | ||
| name: Socket Basics Security Scan | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened] | ||
|
|
||
| jobs: | ||
| socket-basics-security-scan: | ||
| uses: ynab/ynab-sast-scanner/.github/workflows/socket-basics.yml@main | ||
| secrets: | ||
| SOCKET_SECURITY_API_KEY: ${{ secrets.SOCKET_SECURITY_API_KEY }} | ||
| SAST_SUPPRESSIONS_APP_PRIVATE_KEY: ${{ secrets.SAST_SUPPRESSIONS_APP_PRIVATE_KEY }} | ||
| ``` | ||
|
|
||
| Both secrets are organization secrets; nothing per-repo is needed. Name them explicitly as | ||
| inputs - inherited secrets are strictly against policy. | ||
|
|
||
| `pull_request` is the only supported trigger and the workflow fails fast on anything else. | ||
|
|
||
| ## Upgrading socket-basics | ||
|
|
||
| One pin: `SOCKET_BASICS_IMAGE` in `.github/workflows/socket-basics.yml`. The local scan | ||
| script reads that same line, so CI and local scans cannot drift. | ||
|
|
||
| Get the digest from [the GHCR package listing](https://github.com/SocketDev/socket-basics/pkgs/container/socket-basics). | ||
| Pin the multi-arch index digest, not a platform-specific one, so local arm64 runs work from | ||
| the same pin. | ||
|
|
||
| ## Scanning locally | ||
|
|
||
| `scripts/dev/socket-basics.py` scans any repo with the same pinned image that the CI scan uses, | ||
| so a local pass should predict the same in the PR check. | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| ```sh | ||
| # Docker Desktop — bundles the daemon, CLI and Compose, and runs in the background | ||
| brew install --cask docker && open -a Docker | ||
|
|
||
| # gh, authenticated — how the suppression configs are read | ||
| brew install gh && gh auth login | ||
|
|
||
| export SOCKET_SECURITY_API_KEY=<your-key> # from 1Password | ||
| ``` | ||
|
|
||
| Any Docker-compatible runtime works; colima, OrbStack and Rancher Desktop are fine. | ||
| Docker Desktop is only the suggestion because it's a single install with nothing to start by hand. | ||
|
|
||
| Suppressions are read with your own `gh` credentials, so there's no secret to distribute: if | ||
| you can see the suppressions repo, you can scan. | ||
|
|
||
| ### Running it | ||
|
|
||
| ```sh | ||
| scripts/dev/socket-basics.py <path-to-repo> | ||
|
|
||
| # Read suppressions from a branch, to review a suppressions PR before it merges | ||
| scripts/dev/socket-basics.py --suppressions-ref <branch> <path-to-repo> | ||
|
|
||
| # Write artifacts somewhere other than ./.socket-scans | ||
| scripts/dev/socket-basics.py --output-dir <dir> <path-to-repo> | ||
| ``` | ||
|
|
||
| If you use **colima**, note it shares only `$HOME` into its VM by default, so scanning a repo | ||
| or writing artifacts outside your home directory fails with a "config file not found" from | ||
| inside the container. The script warns when it sees this. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.