Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,6 @@ DCO
CITATION.md
CODE_OF_CONDUCT.md
SECURITY.md
THIRD_PARTY.md
design.md

# Dev-only dotfiles
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ All workflows that use `.github/actions/setup-python-env` now default to the ver
| -------------------------------------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------- |
| [ci-checks.yml](ci-checks.yml) | Push to `main`, PRs, manual | Format, typecheck, unit tests, and CPU smoke tests |
| [gpu-tests.yml](gpu-tests.yml) | Nightly, manual | GPU smoke tests (required) and E2E tests |
| [container-build.yml](container-build.yml) | Container/dependency PRs, `v*`, manual | Builds the extra-driven GPU container image and publishes GHCR tags for release tags and same-repo PRs |
| [conventional-commit.yml](conventional-commit.yml) | PRs | Validates PR titles follow conventional commit format |
| [docs.yml](docs.yml) | Push to `main` (docs paths) | Publishes `main` docs as the `latest` GitHub Pages version |
| [release.yml](release.yml) | Push tags to `v*` | Builds and publishes package to Test PyPI/PyPI, creates a GitHub release, and publishes versioned docs |
Expand Down Expand Up @@ -87,6 +88,11 @@ flowchart LR
slackNotify[Slack Notification]
end

subgraph containers [Container Build]
buildContainer[Build cu129 Image]
publishGhcr[Publish GHCR Tags]
end

subgraph internalRelease [Internal Release]
buildWheelInt[Build Wheel]
publishArtifactory[Publish to Artifactory/PyPI]
Expand All @@ -95,10 +101,11 @@ flowchart LR
push --> ci
schedule --> gpu
manual --> ci & gpu
pr --> ci & conventional & secrets
tag[Tag push v[0-9]*] --> release
pr --> ci & conventional & secrets & containers
tag[Tag push v[0-9]*] --> release & containers

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Quote the Mermaid node label -- nested brackets will break the diagram.

tag[Tag push v[0-9]*] contains [ and ] inside the node text. Mermaid terminates the label at the first ], leaving *] to cause a parse/render error. Wrap the label in quotes.

📝 Proposed fix
-    tag[Tag push v[0-9]*] --> release & containers
+    tag["Tag push v[0-9]*"] --> release & containers

As per coding guidelines: "Use Mermaid diagrams with no spaces in node IDs, quote labels with special characters".


buildWheel --> publishPyPI --> ghRelease --> slackNotify
buildContainer --> publishGhcr
buildWheelInt --> publishArtifactory

conventional -.->|reuses| FW-CI-templates
Expand Down
159 changes: 159 additions & 0 deletions .github/workflows/container-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
# Copyright (c) 2024-2026, NVIDIA CORPORATION.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: Container Build

on:
pull_request:
branches:
- main
paths:
- '.dockerignore'
- '.github/workflows/container-build.yml'
- 'containers/**'
- 'Makefile'
- 'pyproject.toml'
- 'uv.lock'
push:
tags:
- 'v*'
workflow_dispatch:

defaults:
run:
shell: bash -x -e -u -o pipefail {0}

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: read
packages: write

env:
REGISTRY_IMAGE: ghcr.io/nvidia-nemo/safe-synthesizer

jobs:
build:
name: Build ${{ matrix.variant }} image
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- variant: cu129
extra: cu129
platforms: linux/amd64
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
Comment thread
kendrickb-nvidia marked this conversation as resolved.
with:
persist-credentials: false
fetch-depth: 0
Comment on lines +53 to +54

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 fetch-depth: 0 fetches the full repository history, but nothing in this workflow consumes it. The version is resolved purely from GITHUB_REF/GITHUB_REF_NAME/GITHUB_SHA (no git describe or similar), and the Docker build context excludes .git/ via .dockerignore. On large repos this meaningfully slows the checkout step.

Suggested change
persist-credentials: false
fetch-depth: 0
persist-credentials: false
fetch-depth: 1

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!


- name: Resolve package version
id: package-version
run: |
if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then
version="${GITHUB_REF_NAME#v}"
else
version="0.0.0+${GITHUB_SHA::12}"
fi
echo "version=${version}" >> "$GITHUB_OUTPUT"

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3

- name: Log in to GHCR
if: >-
${{
github.event_name == 'push' ||
(
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository
)
}}
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract image metadata
id: meta
uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5
with:
images: ${{ env.REGISTRY_IMAGE }}
flavor: |
latest=false
tags: |
type=raw,value=${{ matrix.variant }},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
type=raw,value=latest-${{ matrix.variant }},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
type=ref,event=pr,prefix=pr-,suffix=-${{ matrix.variant }}
type=sha,prefix=sha-,suffix=-${{ matrix.variant }}
type=semver,pattern={{version}}-${{ matrix.variant }}
type=semver,pattern={{major}}.{{minor}}-${{ matrix.variant }}
labels: |
org.opencontainers.image.version=${{ steps.package-version.outputs.version }}
com.nvidia.nemo.safe-synthesizer.extra=${{ matrix.extra }}
com.nvidia.nemo.safe-synthesizer.variant=${{ matrix.variant }}
Comment on lines +84 to +93

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

docker/metadata-action prerelease handling type=raw vs type=semver floating tags

💡 Result:

In the docker/metadata-action, the choice between type=raw and type=semver significantly impacts how pre-release versions are handled, particularly regarding tag floating, prefix stripping, and automatic 'latest' tag generation [1][2]. Type=semver The semver type is designed for automated versioning based on SemVer-compliant Git tags [1]. It includes opinionated, built-in logic for pre-release handling [1]. - Pre-release behavior: When a pre-release (e.g., v1.2.3-alpha) is detected, the action intentionally limits tag generation [1][3]. It will typically only produce the full version string ({{version}} or {{raw}}) and avoids creating broader, mutable 'floating' tags like the major (e.g., v1) or minor (e.g., v1.2) identifiers [1][2]. This is by design to prevent unstable pre-release code from overriding stable production tags [1]. - Prefix handling: By default, type=semver strips the 'v' prefix from tags [4][2]. To preserve the 'v', you should use the prefix attribute (e.g., type=semver,pattern={{version}},prefix=v) rather than hardcoding it into the pattern [4][2]. - Latest tag: For pre-releases, the implicit 'latest' tag generation (if flavor.latest=auto is used) is automatically disabled [5]. Type=raw The raw type provides a way to output custom, explicit tag values [1]. - Pre-release behavior: It does not inherently understand SemVer or distinguish between releases and pre-releases [1]. It simply outputs exactly what is provided in the value attribute [1]. - Floating tags: If you use type=raw to manually create 'floating' tags (e.g., naming your image 'my-app:1'), the action will create these tags regardless of whether the source tag is a pre-release [6]. You must implement your own conditional logic using the enable attribute if you wish to prevent pre-release versions from overriding these tags [3][6]. - Usage: This type is best suited for scenarios where you need full control over the tag naming convention and do not want the action's opinionated semver logic applied [1]. Summary Table Feature | type=semver | type=raw --- | --- | --- Automatic Pre-release Logic | Yes (Limits output to prevent overriding) [1] | No (Literal output) [1] Prefix Handling | Automatically strips 'v' (configurable) [4][2] | None (Literal) [1] Implicit 'latest' Tag | Automatically disabled for pre-releases [5] | Not influenced (can lead to conflicts) [5] Control | Opinionated/Automated [1] | Manual/Explicit [1] To maintain safety during pre-release cycles, prefer type=semver to benefit from the built-in safeguards that prevent unstable builds from inadvertently updating 'latest' or major/minor floating tags [1][5]. Use type=raw only when you have specific, non-standard naming requirements and are prepared to manually manage the conditional logic for when those tags should be applied [6].

Citations:


Prevent RC tags from overwriting the floating cu129 / latest-cu129 tags

In .github/workflows/container-build.yml (lines 84-93), the type=raw tags for value=${{ matrix.variant }} and value=latest-${{ matrix.variant }} are enabled for any refs/tags/v*. Since type=raw doesn’t have prerelease awareness, RC tags (per the repo’s vX.Y.ZrcN scheme) will also generate/overwrite the floating “stable” tags for that variant.

Gate these type=raw entries to exclude prereleases (e.g., tags containing rc):

🔧 Possible approach (exclude RC prereleases from the raw floating tags)
-            type=raw,value=${{ matrix.variant }},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
-            type=raw,value=latest-${{ matrix.variant }},enable=${{ startsWith(github.ref, 'refs/tags/v') }}
+            type=raw,value=${{ matrix.variant }},enable=${{ startsWith(github.ref, 'refs/tags/v') && !contains(github.ref_name, 'rc') }}
+            type=raw,value=latest-${{ matrix.variant }},enable=${{ startsWith(github.ref, 'refs/tags/v') && !contains(github.ref_name, 'rc') }}


- name: Build and push image
if: >-
${{
github.event_name == 'push' ||
(
github.event_name == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.repository
)
}}
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
with:
context: .
file: containers/Dockerfile.cuda
target: runtime
platforms: ${{ matrix.platforms }}
push: true
build-args: |
CONTAINER_EXTRA=${{ matrix.extra }}
CONTAINER_VARIANT=${{ matrix.variant }}
PACKAGE_VERSION=${{ steps.package-version.outputs.version }}
labels: ${{ steps.meta.outputs.labels }}
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=registry,ref=${{ env.REGISTRY_IMAGE }}:buildcache-${{ matrix.variant }}
cache-to: type=registry,ref=${{ env.REGISTRY_IMAGE }}:buildcache-${{ matrix.variant }},mode=max,oci-mediatypes=true,image-manifest=true

- name: Build image
if: >-
${{
github.event_name != 'push' &&
(
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name != github.repository
)
}}
uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6
with:
context: .
file: containers/Dockerfile.cuda
target: runtime
platforms: ${{ matrix.platforms }}
push: false
build-args: |
CONTAINER_EXTRA=${{ matrix.extra }}
CONTAINER_VARIANT=${{ matrix.variant }}
PACKAGE_VERSION=${{ steps.package-version.outputs.version }}
labels: ${{ steps.meta.outputs.labels }}
tags: ${{ steps.meta.outputs.tags }}
3 changes: 3 additions & 0 deletions .mise/tasks/container/build/gpu
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ container_cmd="$(resolve_container_cmd)"
--platform "${CONTAINER_GPU_PLATFORM:-linux/amd64}" \
--tag "${CONTAINER_GPU_IMAGE:-nss-gpu:latest}" \
--target runtime \
--build-arg "CONTAINER_EXTRA=${CONTAINER_GPU_EXTRA:-cu129}" \
--build-arg "CONTAINER_VARIANT=${CONTAINER_GPU_VARIANT:-${CONTAINER_GPU_EXTRA:-cu129}}" \
--build-arg "PACKAGE_VERSION=${CONTAINER_GPU_PACKAGE_VERSION:-}" \
--progress=plain \
-f "${CONTAINER_GPU_FILE:-containers/Dockerfile.cuda}" \
.
3 changes: 3 additions & 0 deletions .mise/tasks/container/build/gpu-dev
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ container_cmd="$(resolve_container_cmd)"
--platform "${CONTAINER_GPU_PLATFORM:-linux/amd64}" \
--tag "${CONTAINER_GPU_IMAGE_DEV:-nss-gpu-dev:latest}" \
--target dev \
--build-arg "CONTAINER_EXTRA=${CONTAINER_GPU_EXTRA:-cu129}" \
--build-arg "CONTAINER_VARIANT=${CONTAINER_GPU_VARIANT:-${CONTAINER_GPU_EXTRA:-cu129}}" \
--build-arg "PACKAGE_VERSION=${CONTAINER_GPU_PACKAGE_VERSION:-}" \
--progress=plain \
-f "${CONTAINER_GPU_FILE:-containers/Dockerfile.cuda}" \
.
3 changes: 3 additions & 0 deletions .mise/tasks/container/build/gpu-multiarch
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ docker buildx build \
--platform linux/amd64,linux/arm64 \
--tag "${CONTAINER_GPU_REGISTRY}/${CONTAINER_GPU_IMAGE:-nss-gpu:latest}" \
--target runtime \
--build-arg "CONTAINER_EXTRA=${CONTAINER_GPU_EXTRA:-cu129}" \
--build-arg "CONTAINER_VARIANT=${CONTAINER_GPU_VARIANT:-${CONTAINER_GPU_EXTRA:-cu129}}" \
--build-arg "PACKAGE_VERSION=${CONTAINER_GPU_PACKAGE_VERSION:-}" \
--progress=plain \
--push \
-f "${CONTAINER_GPU_FILE:-containers/Dockerfile.cuda}" \
Expand Down
2 changes: 1 addition & 1 deletion STYLE_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -765,7 +765,7 @@ Testing conventions are substantial enough to warrant their own section. For the

Two Dockerfiles live in `containers/`:

- [containers/Dockerfile.cuda](containers/Dockerfile.cuda) -- CUDA GPU image (deps/runtime/dev stages). The production reference for these conventions.
- [containers/Dockerfile.cuda](containers/Dockerfile.cuda) -- CUDA GPU image (uv/runtime/dev stages). The production reference for these conventions.
- [containers/Dockerfile.test_ci](containers/Dockerfile.test_ci) -- CPU-only CI image (`mise run test:ci-container`).

See [containers/README.md](containers/README.md) for build arguments and mise tasks.
Expand Down
Loading
Loading