Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 24 additions & 20 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,27 +24,31 @@ jobs:
with:
persist-credentials: false

- name: Validate version and patch k8s manifest
- name: Download backend digest
id: download_backend_digest
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: runtime-digest-${{ github.sha }}-${{ github.run_attempt }}-backend
path: ${{ runner.temp }}/backend-digest
digest-mismatch: error

- name: Download frontend digest
id: download_frontend_digest
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: runtime-digest-${{ github.sha }}-${{ github.run_attempt }}-frontend
path: ${{ runner.temp }}/frontend-digest
digest-mismatch: error

- name: Validate version and render k8s manifests
id: render_manifests
env:
REPO_OWNER: ${{ github.repository_owner }}
run: |
version="$(cat VERSION)"
if [[ ! "$version" =~ ^[0-9]+[.][0-9]+[.][0-9]+([-.][0-9A-Za-z.-]+)?$ ]]; then
printf 'VERSION %s is not a valid release version\n' "$version" >&2
exit 1
fi
repo_owner="$(printf '%s' "$REPO_OWNER" | tr '[:upper:]' '[:lower:]')"
if [[ ! "$repo_owner" =~ ^[a-z0-9][a-z0-9._-]*$ ]]; then
printf 'Repository owner %s is not valid for GHCR image names\n' "$repo_owner" >&2
exit 1
fi
image="ghcr.io/${repo_owner}/ai_email_client-backend:${version}"
sed -E -i "s#ghcr.io/[^[:space:]]+/ai_email_client-backend:REPLACE_ME_VERSION#${image}#" k8s/backend-deployment.yaml
grep -F "$image" k8s/backend-deployment.yaml >/dev/null

frontend_image="ghcr.io/${repo_owner}/ai_email_client-frontend:${version}"
sed -E -i "s#ghcr.io/[^[:space:]]+/ai_email_client-frontend:REPLACE_ME_VERSION#${frontend_image}#" k8s/frontend-deployment.yaml
grep -F "$frontend_image" k8s/frontend-deployment.yaml >/dev/null
backend_digest="$(cat "$RUNNER_TEMP/backend-digest/image-digest.txt")"
frontend_digest="$(cat "$RUNNER_TEMP/frontend-digest/image-digest.txt")"
bash scripts/render_release_manifests.sh "$REPO_OWNER" \
"$backend_digest" "$frontend_digest" "$RUNNER_TEMP/release-manifests"

- name: Setup Kubeconfig
env:
Expand All @@ -62,7 +66,7 @@ jobs:
- name: Apply to AKS
run: |
trap 'rm -f "$KUBECONFIG"' EXIT
kubectl apply -f k8s/backend-deployment.yaml -n naruon-dev
kubectl apply -f "$RUNNER_TEMP/release-manifests/backend-deployment.yaml" -n naruon-dev
kubectl rollout status deployment/backend -n naruon-dev --timeout=120s
kubectl apply -f k8s/frontend-deployment.yaml -n naruon-dev
kubectl apply -f "$RUNNER_TEMP/release-manifests/frontend-deployment.yaml" -n naruon-dev
kubectl rollout status deployment/frontend -n naruon-dev --timeout=120s
21 changes: 21 additions & 0 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,27 @@ jobs:
printf -- '- Digest: %s\n' "$IMAGE_DIGEST"
} >> "$GITHUB_STEP_SUMMARY"

- name: Prepare deployment digest
if: matrix.component == 'backend' || matrix.component == 'frontend'
env:
IMAGE_DIGEST: ${{ steps.build.outputs.digest }}
run: |
if [[ ! "$IMAGE_DIGEST" =~ ^sha256:[0-9a-f]{64}$ ]]; then
printf 'Invalid published image digest\n' >&2
exit 1
fi
mkdir "$RUNNER_TEMP/runtime-digest"
printf '%s\n' "$IMAGE_DIGEST" > "$RUNNER_TEMP/runtime-digest/image-digest.txt"

- name: Upload deployment digest
id: upload_digest
if: matrix.component == 'backend' || matrix.component == 'frontend'
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: runtime-digest-${{ github.sha }}-${{ github.run_attempt }}-${{ matrix.component }}
path: ${{ runner.temp }}/runtime-digest/image-digest.txt
if-no-files-found: error

deploy_preflight:
name: Detect AKS deploy configuration
needs: publish_images
Expand Down
6 changes: 6 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,12 @@ in this repo.

## Release governance defaults

- 배포 이미지의 tag는 게시 artifact의 동일성을 증명하지 않는다. matrix별 digest를
같은 실행·revision·attempt의 개별 artifact로 전달하고 두 runtime의 sha256을
모두 검증한 뒤 `image@sha256` manifest를 생성한다. 누락·변조·잘못된 digest는
credential 설정과 클러스터 변경 전에 실패해야 한다. 원본 manifest와 보안 설정은
보존하고 실제 workflow 생성 명령도 테스트한다. artifact 검증은 readiness,
배포 직렬화, rollback과 실제 사용자 흐름 검증을 대신하지 않는다.
- GitHub Actions used by governed workflows must be pinned to full commit SHAs
with a trailing version comment, for example `# v6`; major-only refs such as
`@v6` are not allowed in release or security workflows.
Expand Down
153 changes: 153 additions & 0 deletions backend/tests/test_release_manifest_digests.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
"""Execute the release manifest contract without registry or cluster access."""

from __future__ import annotations

import os
from pathlib import Path
import shutil
import subprocess

import pytest
import yaml


REPO_ROOT = Path(__file__).resolve().parents[2]
RENDER_SCRIPT = REPO_ROOT / "scripts/render_release_manifests.sh"
BACKEND_DIGEST = "sha256:" + "a" * 64
FRONTEND_DIGEST = "sha256:" + "b" * 64


def run_renderer(tmp_path: Path, backend_digest: str, frontend_digest: str) -> subprocess.CompletedProcess[str]:
"""Render checked-in manifests in an isolated directory using unit digests."""
shutil.copytree(REPO_ROOT / "k8s", tmp_path / "k8s")
shutil.copyfile(REPO_ROOT / "VERSION", tmp_path / "VERSION")
return subprocess.run(
["bash", str(RENDER_SCRIPT), "ContextualWisdomLab", backend_digest, frontend_digest, "rendered"],
cwd=tmp_path,
env={"PATH": os.defpath},
text=True,
capture_output=True,
check=False,
)


def test_release_renderer_binds_both_images_without_changing_source(tmp_path: Path) -> None:
"""Keep all pod security/configuration fields while replacing only image refs."""
result = run_renderer(tmp_path, BACKEND_DIGEST, FRONTEND_DIGEST)
assert result.returncode == 0, result.stderr
for component, digest in (("backend", BACKEND_DIGEST), ("frontend", FRONTEND_DIGEST)):
source_path = tmp_path / "k8s" / f"{component}-deployment.yaml"
assert source_path.read_bytes() == (REPO_ROOT / "k8s" / source_path.name).read_bytes()
expected = yaml.safe_load(source_path.read_text())
expected["spec"]["template"]["spec"]["containers"][0]["image"] = (
f"ghcr.io/contextualwisdomlab/ai_email_client-{component}@{digest}"
)
rendered = yaml.safe_load((tmp_path / "rendered" / source_path.name).read_text())
assert rendered == expected


@pytest.mark.parametrize("invalid_digest", ["", "latest", "sha256:" + "a" * 63, "sha256:" + "A" * 64, BACKEND_DIGEST + "\ninjected", "$(touch injected)"])
@pytest.mark.parametrize("invalid_component", ["backend", "frontend"])
def test_release_renderer_rejects_either_invalid_digest_before_output(
tmp_path: Path, invalid_digest: str, invalid_component: str
) -> None:
"""Reject absent or malformed identities before creating either manifest."""
result = run_renderer(
tmp_path,
invalid_digest if invalid_component == "backend" else BACKEND_DIGEST,
invalid_digest if invalid_component == "frontend" else FRONTEND_DIGEST,
)
assert result.returncode != 0
assert "Invalid image digest" in result.stderr
assert not (tmp_path / "rendered").exists()
assert not (tmp_path / "injected").exists()


@pytest.mark.parametrize("failure_case", ["backend_missing", "frontend_missing", "backend_duplicate", "frontend_duplicate", "invalid_version", "existing_output"])
def test_release_renderer_preserves_prior_output_on_source_drift(
tmp_path: Path, failure_case: str
) -> None:
"""Fail before publishing a second manifest set or overwriting prior evidence."""
first_result = run_renderer(tmp_path, BACKEND_DIGEST, FRONTEND_DIGEST)
assert first_result.returncode == 0, first_result.stderr
prior_output = {path.name: path.read_bytes() for path in (tmp_path / "rendered").iterdir()}
if failure_case == "invalid_version":
(tmp_path / "VERSION").write_text("not-a-version\n")
elif failure_case != "existing_output":
component, mutation = failure_case.split("_")
source_path = tmp_path / "k8s" / f"{component}-deployment.yaml"
source_text = source_path.read_text()
image_line = next(line for line in source_text.splitlines() if "image: ghcr.io/" in line)
source_path.write_text(source_text.replace(image_line, "" if mutation == "missing" else image_line + "\n" + image_line))
output_directory = "rendered" if failure_case == "existing_output" else "next_rendered"
result = subprocess.run(
["bash", str(RENDER_SCRIPT), "ContextualWisdomLab", BACKEND_DIGEST, FRONTEND_DIGEST, output_directory],
cwd=tmp_path,
env={"PATH": os.defpath},
capture_output=True,
text=True,
check=False,
)
assert result.returncode != 0
assert not (tmp_path / "next_rendered").exists()
assert {path.name: path.read_bytes() for path in (tmp_path / "rendered").iterdir()} == prior_output


def test_release_workflow_passes_separate_same_revision_artifacts() -> None:
"""Bind producer and deployment consumer without matrix output overwrites."""
publish = yaml.safe_load((REPO_ROOT / ".github/workflows/docker-publish.yml").read_text())
deploy = yaml.safe_load((REPO_ROOT / ".github/workflows/deploy.yml").read_text())
upload = next(step for step in publish["jobs"]["publish_images"]["steps"] if step.get("id") == "upload_digest")
assert "${{ matrix.component }}" in upload["with"]["name"]
assert "${{ github.sha }}" in upload["with"]["name"]
assert "${{ github.run_attempt }}" in upload["with"]["name"]
assert upload["with"]["if-no-files-found"] == "error"
steps = deploy["jobs"]["deploy"]["steps"]
for component in ("backend", "frontend"):
download = next(step for step in steps if step.get("id") == f"download_{component}_digest")
assert download["with"]["name"] == upload["with"]["name"].replace("${{ matrix.component }}", component)
assert download["with"]["digest-mismatch"] == "error"
assert "github-token" not in download["with"]
render = next(step for step in steps if step.get("id") == "render_manifests")
assert "scripts/render_release_manifests.sh" in render["run"]
apply_step = next(step for step in steps if step["name"] == "Apply to AKS")
assert '"$RUNNER_TEMP/release-manifests/backend-deployment.yaml"' in apply_step["run"]
assert '"$RUNNER_TEMP/release-manifests/frontend-deployment.yaml"' in apply_step["run"]


@pytest.mark.parametrize("artifact_state", ["valid", "missing_backend", "missing_frontend", "invalid_backend", "invalid_frontend"])
def test_deployment_executes_renderer_only_with_both_valid_artifacts(
tmp_path: Path, artifact_state: str
) -> None:
"""Execute the real workflow shell before any credential or cluster step."""
shutil.copytree(REPO_ROOT / "k8s", tmp_path / "k8s")
(tmp_path / "scripts").mkdir()
shutil.copyfile(RENDER_SCRIPT, tmp_path / "scripts" / RENDER_SCRIPT.name)
shutil.copyfile(REPO_ROOT / "VERSION", tmp_path / "VERSION")
runner_temp = tmp_path / "runner_temp"
runner_temp.mkdir()
for component, digest in (("backend", BACKEND_DIGEST), ("frontend", FRONTEND_DIGEST)):
digest_directory = runner_temp / f"{component}-digest"
digest_directory.mkdir()
if artifact_state != f"missing_{component}":
(digest_directory / "image-digest.txt").write_text(
"invalid\n" if artifact_state == f"invalid_{component}" else digest + "\n"
)
workflow = yaml.safe_load((REPO_ROOT / ".github/workflows/deploy.yml").read_text())
render_step = next(step for step in workflow["jobs"]["deploy"]["steps"] if step.get("id") == "render_manifests")
result = subprocess.run(
["bash", "-euo", "pipefail", "-c", render_step["run"]],
cwd=tmp_path,
env={"PATH": os.defpath, "RUNNER_TEMP": str(runner_temp), "REPO_OWNER": "ContextualWisdomLab"},
capture_output=True,
text=True,
check=False,
)
if artifact_state == "valid":
assert result.returncode == 0, result.stderr
for component, digest in (("backend", BACKEND_DIGEST), ("frontend", FRONTEND_DIGEST)):
manifest = yaml.safe_load((runner_temp / "release-manifests" / f"{component}-deployment.yaml").read_text())
assert manifest["spec"]["template"]["spec"]["containers"][0]["image"].endswith("@" + digest)
else:
assert result.returncode != 0
assert not (runner_temp / "release-manifests").exists()
55 changes: 54 additions & 1 deletion docs/doctoring/runtime-image-boundary-verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,60 @@ uv run --frozen --offline python -m pytest --noconftest \
frontend SIGTERM, AMD64·통합 이미지, 배포·rollback, 인증된 제품 화면의 Visual
Inspection은 별도 근거가 필요하다. 문서의 시각 검수도 제품 검수를 대신하지 않는다.

## 참고 문헌
## 배포 digest 전달 후속 수리

상태: Proposed. 이 절은 `9b137f25f426743e18fc61125575ec7949d45db8` 위의
후속 작업이며 앞선 이미지 빌드 기록의 검증 범위를 확대하지 않는다.
[이슈 #1022의 Linux 재현](https://github.com/ContextualWisdomLab/naruon/issues/1022#issuecomment-5564923743)에서
기존 manifest 생성 명령은 digest 없이 버전 tag로 두 이미지를 선택했다.
운영자가 같은 tag를 다른 이미지에 붙이면 배포 결과와 게시 기록이 달라질 수 있다.
외부 registry의 tag 변경 방지 설정은 이번 조사에서 확인하지 않았다.

선택한 계약은 publisher의 실제 `steps.build.outputs.digest`를 backend/frontend
각각의 artifact로 전달하는 것이다. 이름에 source SHA, 실행 attempt, component를
포함하며 같은 실행의 deploy job만 내려받는다. 공통 matrix output은 완료 순서에
따라 서로 덮어쓸 수 있어 쓰지 않는다. 다운로드 무결성 불일치는 오류로 처리한다.
누락된 artifact를 이전 실행에서 검색하거나 tag로 되돌리는 fallback은 없다.
실패 job만 재실행해 현재 attempt의 두 artifact가 모두 없으면 배포를 거부한다.
재시도는 보호 source와 버전을 재검증한 뒤 게시 matrix 전체를 포함해야 한다.

`scripts/render_release_manifests.sh`는 기존 manifest의 image 필드만 치환한다.
두 digest의 `sha256:` 및 소문자 64자리 hex, repository owner, VERSION, 두 원본의
단일 placeholder를 먼저 확인한다. 별도 출력 디렉터리에 두 manifest를 만들고
deploy workflow는 이 파일만 apply한다. GNU 전용 `sed -i` 대신 stdout 출력을
사용해 macOS와 Linux의 같은 명령을 검증한다. Python 제품 runtime은 추가하지
않았고 기존 pytest는 shell과 workflow 계약의 테스트 도구로만 사용한다.

검증 명령은 다음과 같다. 합성 digest는 unit test 입력일 뿐 실제 게시 증거가 아니다.

```sh
uv run --project backend --frozen --offline python -m pytest --noconftest \
backend/tests/test_release_manifest_digests.py \
backend/tests/test_runtime_image_targets.py \
backend/tests/test_release_governance.py -q -W error
actionlint .github/workflows/deploy.yml .github/workflows/docker-publish.yml
shellcheck scripts/render_release_manifests.sh
```

새 계약의 초기 14개 RED는 renderer와 artifact 전달 부재를 확인했다. 구현 후
기존 검사 포함 52개가 통과했고, 실제 workflow shell의 정상·backend/frontend
누락·잘못된 값 5개를 추가한 결과 57 passed, 4.32초, exit 0이었다.
최종 commit과 이후 검증은 해당 PR 기록에 연결한다.

이 변경만으로 자동 배포가 적격해지지는 않는다. #1365 보호 통합, #1562의
concurrency 변경과 #1583 action pin의 delta 보존, 실제 artifact 업로드·다운로드,
환경 승인·release 직렬화·readiness·부분 배포 복구·실제 imageID·정상 인증 흐름은
별도 수용 기준이다. 아직 tag 발행이나 클러스터 변경을 실행하지 않았다.

### 후속 계약의 근거

GitHub. (n.d.-a). *Upload a build artifact* [Action definition].
https://github.com/actions/upload-artifact/blob/043fb46d1a93c77aae656e7c1c64a875d1fc6a0a/action.yml

GitHub. (n.d.-b). *Download a build artifact* [Action definition].
https://github.com/actions/download-artifact/blob/3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c/action.yml

## 이미지 경계 참고 문헌

Docker, Inc. (n.d.). *JSONArgsRecommended*. Docker Docs.
https://docs.docker.com/reference/build-checks/json-args-recommended/
Expand Down
44 changes: 44 additions & 0 deletions scripts/render_release_manifests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# Render only the two checked-in runtime manifests; never contact a cluster.
set -euo pipefail

if [[ "$#" != 4 ]]; then
printf 'Expected repository owner, backend digest, frontend digest, output directory\n' >&2
exit 1
fi
repo_owner="$(printf '%s' "$1" | tr '[:upper:]' '[:lower:]')"
backend_digest="$2"
frontend_digest="$3"
output_directory="$4"
if [[ ! "$repo_owner" =~ ^[a-z0-9][a-z0-9._-]*$ ]]; then
printf 'Invalid repository owner\n' >&2
exit 1
fi
for image_digest in "$backend_digest" "$frontend_digest"; do
if [[ ! "$image_digest" =~ ^sha256:[0-9a-f]{64}$ ]]; then
printf 'Invalid image digest\n' >&2
exit 1
fi
done
release_version="$(cat VERSION)"
if [[ ! "$release_version" =~ ^[0-9]+[.][0-9]+[.][0-9]+([-.][0-9A-Za-z.-]+)?$ ]]; then
printf 'Invalid release version\n' >&2
exit 1
fi
# Validate both sources before writing either result; preserve source manifests.
for image_component in backend frontend; do
image_pattern="^([[:space:]]*image: )ghcr[.]io/[A-Za-z0-9._-]+/ai_email_client-${image_component}:REPLACE_ME_VERSION$"
match_count="$(grep -Ec "$image_pattern" "k8s/${image_component}-deployment.yaml" || true)"
if [[ "$match_count" != 1 ]]; then
printf 'Expected one runtime image placeholder per manifest\n' >&2
exit 1
fi
done
mkdir "$output_directory"
for image_component in backend frontend; do
image_digest="$backend_digest"
if [[ "$image_component" == frontend ]]; then image_digest="$frontend_digest"; fi
image_pattern="^([[:space:]]*image: )ghcr[.]io/[A-Za-z0-9._-]+/ai_email_client-${image_component}:REPLACE_ME_VERSION$"
sed -E "s#${image_pattern}#\1ghcr.io/${repo_owner}/ai_email_client-${image_component}@${image_digest}#" \
"k8s/${image_component}-deployment.yaml" > "$output_directory/${image_component}-deployment.yaml"
done