Skip to content

build(docker): publish a multi-arch (amd64 + arm64) image - #425

Open
jackrieck wants to merge 1 commit into
solana-foundation:mainfrom
jackrieck:build/multi-arch-docker-image
Open

build(docker): publish a multi-arch (amd64 + arm64) image#425
jackrieck wants to merge 1 commit into
solana-foundation:mainfrom
jackrieck:build/multi-arch-docker-image

Conversation

@jackrieck

Copy link
Copy Markdown

Problem

ghcr.io/solana-foundation/pay:latest is a single-architecture amd64 image, not
a manifest list, so it cannot run on arm64 hosts — Graviton, Fargate arm64, or
Apple Silicon without emulation.

$ docker manifest inspect --verbose ghcr.io/solana-foundation/pay:latest
{
  "Ref": "ghcr.io/solana-foundation/pay:latest",
  "Descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "digest": "sha256:bd8bd9f94e408501d1b57f12a36908b08482faa9523663c6f911105b3aa5d1ec",
    "platform": {
      "architecture": "amd64",
      "os": "linux"
    }
  },
  ...

A multi-arch image would return application/vnd.oci.image.index.v1+json with a
manifests array. This returns one manifest, pinned to amd64.

The cause is that .github/workflows/docker.yml passes no platforms: to
docker/build-push-action, so the build inherits the ubuntu-latest runner's
architecture.

arm64 is clearly a supported target — release-cli.yml already ships
pay-aarch64-unknown-linux-gnu.tar.gz on every release. Only the container
misses it.

Approach

Each architecture builds on a runner of that architecture, and a merge job
combines the two digests into one manifest list. This is Docker's documented
multi-runner distributed build
pattern.

The obvious one-line fix — adding platforms: linux/amd64,linux/arm64 on a
single amd64 runner — is not viable here. rust/Dockerfile compiles the
workspace from source (cargo build --release --features gcp_kms,redis-session-store,
with cmake and libclang), and the arm64 half of that build would run under QEMU
emulation. The current native build takes ~10 minutes; the emulated one would
be measured in hours and could plausibly hit the 6-hour job ceiling.

Native runners avoid that entirely, and cost nothing here: ubuntu-24.04-arm is
a standard GitHub-hosted runner, and
"use of the standard GitHub-hosted runners is free and unlimited on public
repositories"
.
The two builds run in parallel, so wall-clock time is roughly unchanged.

Changes

.github/workflows/docker.yml only.

  • build becomes a two-entry matrix: linux/amd64 on ubuntu-latest,
    linux/arm64 on ubuntu-24.04-arm. No QEMU, no setup-qemu-action.
  • Each build pushes by digest (push-by-digest=true,name-canonical=true) and
    uploads the digest as an artifact.
  • A new merge job runs docker buildx imagetools create to assemble the
    digests into one manifest list, then inspects the result.
  • The docker/metadata-action tag configuration moves to the merge job
    unchanged — including type=match,pattern=^pay-v(.*)$,group=1 and
    latest. Tags belong to the manifest list, not to either single-arch image.
    The per-arch job keeps a metadata-action step for labels only.
  • The workflow_dispatch push input behaves as before: a dry run still builds
    both architectures (that being the point of it) and writes nothing to GHCR.
    Its outputs resolves to type=cacheonly and the merge job is skipped.

Verification

  • actionlint 1.7.7 (which runs shellcheck over run: blocks) reports no
    findings for the modified workflow.

  • rust/Dockerfile builds natively for linux/arm64 unmodified — verified
    locally on Apple Silicon with docker build --platform linux/arm64 ./rust,
    which is the same unemulated path ubuntu-24.04-arm takes. cargo build --release --features gcp_kms,redis-session-store finished in 4m32s, and the
    resulting image runs:

    $ docker run --rm --platform linux/arm64 <image> --version
    pay 0.27.0
    
  • The merge job's imagetools create invocation was exercised against a
    recorded DOCKER_METADATA_OUTPUT_JSON for a pay-v0.27.0 tag push, and
    assembles the expected command:

    docker buildx imagetools create \
      -t ghcr.io/solana-foundation/pay:0.27.0 \
      -t ghcr.io/solana-foundation/pay:pay-v0.27.0 \
      -t ghcr.io/solana-foundation/pay:latest \
      ghcr.io/solana-foundation/pay@sha256:<amd64> \
      ghcr.io/solana-foundation/pay@sha256:<arm64>
    

I could not do a full end-to-end dispatch on my fork: GitHub disables Actions on
forks until a maintainer enables them in the UI, and I have no way to click that
from here. Happy to run one if that would help — or a workflow_dispatch with
push: false on this branch would exercise both builds without touching the
registry.

🤖 Generated with Claude Code

`ghcr.io/solana-foundation/pay:latest` is a single amd64 manifest, not a
manifest list, so it cannot run on arm64 hosts (Graviton, Fargate arm64,
Apple Silicon without emulation) even though the release workflow already
ships `pay-aarch64-unknown-linux-gnu`.

Build each architecture on a runner of that architecture and merge the two
digests into one manifest list. QEMU is not an option here: `rust/Dockerfile`
compiles the workspace from source, and emulating that build would take hours.
`ubuntu-24.04-arm` is free for public repositories, so the extra runner costs
nothing.

The `docker/metadata-action` tag configuration moves to the merge job unchanged
— tags belong to the manifest list, not to either single-arch image. The
`workflow_dispatch` dry run still builds both architectures and pushes nothing.
@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

@jackrieck is attempting to deploy a commit to the Solana Foundation Team on Vercel.

A member of the Team first needs to authorize it.

@greptile-apps

greptile-apps Bot commented Aug 10, 2026

Copy link
Copy Markdown

Greptile Summary

This change builds amd64 and arm64 container images on native runners, publishes their digest references, and merges them into one multi-architecture GHCR manifest. The manual push=true path was exercised on a non-release ref: docker/metadata-action@v5 emits latest as the image version, so the inspection command resolves to a valid :latest reference. The reported empty-tag inspection failure was therefore disproved. The pull request cannot be merged until its unsigned commit is replaced with a verified commit.

Confidence Score: 3/5

Do not merge until the commit is recreated or amended with a verified signature.

Git verification was executed against the exact commit and confirmed that it has no signature. The reported workflow inspection failure was directly disproved by exercising the relevant manual-dispatch metadata path.

Files Needing Attention: The commit associated with this pull request needs a verified signature; no source-file change is needed for the disproved Docker inspection concern.

Security Review

The pull request commit is unsigned. This prevents cryptographic verification of the change author and violates the repository requirement that commits be signed and verified before merge.

T-Rex T-Rex Logs

What T-Rex did

  • Ran the verification script for the requested commit signature and reviewed the unsigned verification output.
  • Validated contract behavior when Docker is absent by using the real metadata-action distribution and a minimal local Docker CLI stand-in, and confirmed the main capture shows Docker image version latest, tag ghcr.io/solana-foundation/pay:latest, and argument acceptance.
  • Generated an additional proof for a posted P1 finding.
  • Observed the verifier exited with code 1 and git show reported signature_status=N with no signer or signing key; saved script and captured output provide reproducible evidence for the exact commit.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (1)

  1. General comment

    P1 Commit 62d4270 is unsigned, violating the verified-commit policy

    • Bug
      • The repository's instruction-0 says to request changes when commits are not signed (verified). The exact commit 62d427072c002b70a9a9731cf9b1366755082adb has Git signature status N; git verify-commit --raw exited with status 1 and neither signer nor signing key was present.
    • Cause
      • The commit object does not contain a cryptographic signature that Git can verify.
    • Fix
      • Amend or recreate the commit with an approved signing key, then verify it with git verify-commit --raw 62d427072c002b70a9a9731cf9b1366755082adb (using the resulting rewritten commit ID).

    T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "build(docker): publish a multi-arch (amd..." | Re-trigger Greptile


env:
IMAGE: ghcr.io/solana-foundation/pay
# A dry run (workflow_dispatch with push=false) still builds every

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Commit signature verification is missing

Commit 62d427072c002b70a9a9731cf9b1366755082adb is unsigned, so this pull request does not meet the repository requirement for signed and verified commits. Recreate or amend the commit with an approved signing key before merging.

Context Used: Request changes if the commits are not signed (ver... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Artifacts

Executed script that verifies the requested commit signature

  • The exact Bash source executed against commit 62d4270, showing the reproducible Git verification commands.

Observed unsigned commit verification output

  • Captured output from the executed verification script: Git verification exits 1 and reports signature status N, confirming the commit is unsigned.

View artifacts

T-Rex Ran code and verified through T-Rex

@jackrieck

Copy link
Copy Markdown
Author

Ran the dispatch offered above on a fork, since docker.yml doesn't trigger on PRs. Both architectures built natively and the merge produced a real manifest list.

Run: https://github.com/jackrieck/pay/actions/runs/31407993608

Build linux/arm64:   success
Build linux/amd64:   success
Merge manifest list: success

The published result is an OCI image index rather than a single manifest:

$ docker buildx imagetools inspect ghcr.io/jackrieck/pay:latest
MediaType: application/vnd.oci.image.index.v1+json

Manifests:
  ...@sha256:d2362205...  Platform: linux/arm64
  ...@sha256:1c4f89ba...  Platform: linux/amd64
  (+ attestation manifests for each)

For contrast, the same command against the current published image returns application/vnd.docker.distribution.manifest.v2+json with "architecture": "amd64" and no Manifests list at all.

And the arm64 image resolved from that index runs:

$ docker run --rm --platform linux/arm64 ghcr.io/jackrieck/pay:latest --version
pay 0.27.0

No QEMU anywhere in the run — each arch compiled on a runner of its own architecture, in parallel.

One incidental finding while packaging this image downstream, in case it's useful: the two Linux release binaries have different glibc floors. pay-x86_64-unknown-linux-gnu needs GLIBC_2.38/2.39, while pay-aarch64-unknown-linux-gnu runs on 2.36. So a Debian bookworm base works for aarch64 and fails at startup for x86_64 with version `GLIBC_2.38' not found. Not something this PR touches — rust/Dockerfile builds from source so it's unaffected — but it may be worth a note for anyone repackaging the released binaries.

@lgalabru

Copy link
Copy Markdown
Collaborator

@jackrieck thanks for your contribution! could you please address greptiles feedback and make sure we're reaching a 5/5 confidence score? thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants