Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 59 additions & 1 deletion .github/workflows/backend-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ jobs:
VULNFLOW_REPORTS_VOLUME: vulnflow-ci-reports
VULNFLOW_AGENT_VOLUME: vulnflow-ci-agent
VULNFLOW_AGENT_TARGETS_FILE: /tmp/vulnflow-targets.yml
VULNFLOW_AGENT_REGISTRY_AUTH_DIR: /tmp/vulnflow-agent-registry
VULNFLOW_AGENT_ID: ci-agent
VULNFLOW_BACKEND_IMAGE: example.invalid/vulnflow-backend:ci
VULNFLOW_AGENT_IMAGE: example.invalid/vulnflow-agent:ci
Expand Down Expand Up @@ -146,6 +147,10 @@ jobs:
assert agent["environment"]["TMPDIR"] == "/var/lib/vulnflow-agent/tmp"
assert agent["read_only"] is True
assert all("docker.sock" not in str(volume) for volume in agent["volumes"])
assert agent["environment"]["DOCKER_CONFIG"] == "/run/vulnflow-registry"
assert any(volume.get("source") == "/tmp/vulnflow-agent-registry"
and volume.get("target") == "/run/vulnflow-registry"
and volume.get("read_only") is True for volume in agent["volumes"])
assert backend["environment"]["SPRING_PROFILES_ACTIVE"] == "prod"
assert "AWS_PROFILE" not in backend["environment"]
assert web["read_only"] is True
Expand Down Expand Up @@ -307,6 +312,8 @@ jobs:
VPS_DEPLOY_PATH: ${{ vars.VPS_DEPLOY_PATH }}
GHCR_USERNAME: ${{ vars.GHCR_USERNAME }}
GHCR_TOKEN: ${{ github.token }}
GHCR_AGENT_USERNAME: ${{ vars.GHCR_AGENT_USERNAME }}
GHCR_AGENT_READ_TOKEN: ${{ secrets.GHCR_AGENT_READ_TOKEN }}
VPS_SSH_PRIVATE_KEY: ${{ secrets.VPS_SSH_PRIVATE_KEY }}
VPS_SSH_KNOWN_HOSTS: ${{ secrets.VPS_SSH_KNOWN_HOSTS }}

Expand Down Expand Up @@ -343,6 +350,8 @@ jobs:
echo "GHCR_USERNAME is invalid"
exit 1
fi
[[ "$GHCR_AGENT_USERNAME" =~ ^[A-Za-z0-9_.-]+$ ]] || { echo "GHCR_AGENT_USERNAME is missing or invalid"; exit 1; }
[[ -n "$GHCR_AGENT_READ_TOKEN" ]] || { echo "GHCR_AGENT_READ_TOKEN is missing"; exit 1; }

- name: Prepare pinned SSH identity
shell: bash
Expand Down Expand Up @@ -403,6 +412,55 @@ jobs:
"$VPS_USER@$VPS_HOST" \
docker login ghcr.io --username "$GHCR_USERNAME" --password-stdin

- name: Install dedicated Agent registry credential
shell: bash
env:
AGENT_IMAGE: ${{ needs.publish_images.outputs.agent_image }}
run: |
python - <<'PY'
import base64
import json
import os
from pathlib import Path

username = os.environ["GHCR_AGENT_USERNAME"]
token = os.environ["GHCR_AGENT_READ_TOKEN"]
encoded = base64.b64encode(f"{username}:{token}".encode()).decode()
target = Path(os.environ["RUNNER_TEMP"]) / "vulnflow-agent-registry.json"
target.write_text(json.dumps({"auths": {"ghcr.io": {"auth": encoded}}}), encoding="utf-8")
target.chmod(0o600)
PY

scp -i "$RUNNER_TEMP/vulnflow-ssh/id" \
-o IdentitiesOnly=yes \
-o StrictHostKeyChecking=yes \
-o UserKnownHostsFile="$RUNNER_TEMP/vulnflow-ssh/known_hosts" \
-P "$VPS_PORT" \
"$RUNNER_TEMP/vulnflow-agent-registry.json" \
"$VPS_USER@$VPS_HOST:$VPS_DEPLOY_PATH/.agent-registry.next.json"

ssh -i "$RUNNER_TEMP/vulnflow-ssh/id" \
-o IdentitiesOnly=yes \
-o StrictHostKeyChecking=yes \
-o UserKnownHostsFile="$RUNNER_TEMP/vulnflow-ssh/known_hosts" \
-p "$VPS_PORT" \
"$VPS_USER@$VPS_HOST" \
bash -s -- "$VPS_DEPLOY_PATH" "$AGENT_IMAGE" <<'REMOTE'
set -euo pipefail
deploy_root=$1
agent_image=$2
trap 'rm -f "$deploy_root/.agent-registry.next.json"' EXIT
docker pull "$agent_image" >/dev/null
docker run --rm --user 0:0 \
-v "$deploy_root/runtime:/runtime" \
-v "$deploy_root/.agent-registry.next.json:/input/config.json:ro" \
--entrypoint /bin/sh "$agent_image" -c '
set -eu
install -d -m 700 -o 10001 -g 10001 /runtime/agent-registry
install -m 600 -o 10001 -g 10001 /input/config.json /runtime/agent-registry/config.json
'
REMOTE

- name: Deploy and verify release
shell: bash
run: |
Expand All @@ -417,4 +475,4 @@ jobs:
- name: Remove temporary credentials
if: always()
shell: bash
run: rm -rf "$RUNNER_TEMP/vulnflow-ssh" "$RUNNER_TEMP/vulnflow-release.env"
run: rm -rf "$RUNNER_TEMP/vulnflow-ssh" "$RUNNER_TEMP/vulnflow-release.env" "$RUNNER_TEMP/vulnflow-agent-registry.json"
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## Unreleased

- Give the production Agent a dedicated read-only Docker configuration for scanning private GHCR images without sharing the host Docker login or socket.
- Present executable targets and result assets as one image catalog in the console while preserving their separate persistence responsibilities.
- Resolve and relink the matching asset atomically whenever an administrator registers or changes an image reference.
- Replace the 10,000-item in-memory findings search with database pagination and bounded DynamoDB cursor batches, avoiding repeated summary reads after the first batch.
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,10 @@ key, synchronizes only versioned deployment files, and never overwrites the
VPS-only runtime environment or target file. Named PostgreSQL, report, and
agent-outbox volumes are preserved.

Private GHCR scans use a separate `read:packages` credential mounted read-only as
the Agent's Docker configuration. The Agent never receives the host Docker login,
deployment token, or Docker socket.

See [the VPS deployment runbook](docs/operations/vps-deployment.md) for one-time preparation,
GitHub variables and secrets, Nginx/TLS, exact-SHA deployment, health checks,
automatic rollback, emergency recovery, and the deployment pause switch. No
Expand Down
1 change: 1 addition & 0 deletions deploy/.env.prod.example
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ VULNFLOW_AGENT_VOLUME=vulnflow-agent-data

# This absolute host path is mounted read-only in the agent container.
VULNFLOW_AGENT_TARGETS_FILE=/srv/vulnflow/runtime/targets.yml
VULNFLOW_AGENT_REGISTRY_AUTH_DIR=/srv/vulnflow/runtime/agent-registry
VULNFLOW_AGENT_ID=vps-production-01
VULNFLOW_SCAN_INTERVAL=6h
VULNFLOW_AGENT_MAX_CONCURRENT_SCANS=1
Expand Down
6 changes: 6 additions & 0 deletions deploy/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ lock_file="${deploy_root}/.deploy.lock"
health_check="${script_dir}/health-check.sh"
compose_file="${script_dir}/docker-compose.prod.yml"
aws_compose_file="${script_dir}/docker-compose.aws.yml"
agent_registry_auth_dir=$(sed -n 's/^VULNFLOW_AGENT_REGISTRY_AUTH_DIR=//p' "${runtime_env}" 2>/dev/null || true)

for command in docker flock realpath; do
command -v "${command}" >/dev/null 2>&1 || { echo "Required command is unavailable: ${command}" >&2; exit 69; }
Expand All @@ -30,6 +31,11 @@ if [[ ! -f "${candidate_release}" ]]; then
echo "Candidate release file is missing." >&2
exit 65
fi
if [[ -z "${agent_registry_auth_dir}" || "${agent_registry_auth_dir}" != /* \
|| ! -f "${agent_registry_auth_dir}/config.json" ]]; then
echo "Agent registry credential is missing or invalid." >&2
exit 65
fi

candidate_real=$(realpath "${candidate_release}")
if [[ "${candidate_real}" != "${deploy_root}/.release.next.env" ]]; then
Expand Down
5 changes: 5 additions & 0 deletions deploy/docker-compose.prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ services:
VULNFLOW_AGENT_COMMANDS_ENABLED: ${VULNFLOW_AGENT_COMMANDS_ENABLED:-false}
VULNFLOW_AGENT_COMMAND_POLL_INTERVAL: ${VULNFLOW_AGENT_COMMAND_POLL_INTERVAL:-5s}
VULNFLOW_TARGETS_FILE: /etc/vulnflow-agent/targets.yml
DOCKER_CONFIG: /run/vulnflow-registry
TRIVY_CACHE_DIR: /var/lib/vulnflow-agent/trivy-cache
TMPDIR: /var/lib/vulnflow-agent/tmp
volumes:
Expand All @@ -108,6 +109,10 @@ services:
source: ${VULNFLOW_AGENT_TARGETS_FILE:?VULNFLOW_AGENT_TARGETS_FILE is required}
target: /etc/vulnflow-agent/targets.yml
read_only: true
- type: bind
source: ${VULNFLOW_AGENT_REGISTRY_AUTH_DIR:?VULNFLOW_AGENT_REGISTRY_AUTH_DIR is required}
target: /run/vulnflow-registry
read_only: true
networks:
- internal
read_only: true
Expand Down
9 changes: 6 additions & 3 deletions docs/operations/on-demand-scans.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,13 @@ need an entry in local `targets.yml`. That file is exclusively the recurring sca
schedule and may contain `targets: []` for command-only operation. Registering a
console target never schedules it automatically. Existing schedules are preserved.

Register an exact image reference under Targets, then choose its name and an Agent
Register an exact image reference under Images, then choose its name and an Agent
under Scans and press Launch scan. Selecting an image alone does not launch work.
Private images require registry credentials accessible to Trivy on the selected
machine. No registry credentials are sent through the console.
Private GHCR images use the selected Agent's dedicated `read:packages` credential,
mounted read-only as its Docker configuration. No registry credential is sent
through the console, stored in the image catalog, inherited from the host Docker
login, or exposed through the Docker socket. Other private registries require an
equivalent Agent-side credential design before their references are enabled.

All successfully ingested reports appear under Results, including scheduled Agent
scans and direct uploads that have no console request. Assets link to their complete
Expand Down
13 changes: 10 additions & 3 deletions docs/operations/vps-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,25 +86,32 @@ values as `production` environment variables:
| `VPS_USER` | Dedicated non-root deploy user |
| `VPS_DEPLOY_PATH` | Absolute VulnFlow directory, for example `/srv/vulnflow` |
| `GHCR_USERNAME` | Optional GHCR user for private packages |
| `GHCR_AGENT_USERNAME` | GitHub account used by the Agent's dedicated read-only package credential |

Configure these environment secrets:

| Secret | Purpose |
| --- | --- |
| `VPS_SSH_PRIVATE_KEY` | Private key dedicated to the deploy user |
| `VPS_SSH_KNOWN_HOSTS` | Pinned host-key line for the configured host and port |
| `GHCR_TOKEN` | Optional read-only package token for private GHCR images |
| `GHCR_AGENT_READ_TOKEN` | Dedicated classic PAT with `read:packages` only, used by Trivy for private image scans |

Obtain the SSH public host key through a trusted channel and verify its
fingerprint with the VPS provider or an existing trusted session. Store the
complete known-hosts line. For a non-default port its host field must use
`[host]:port`. The workflow uses `StrictHostKeyChecking=yes`; it never trusts
the result of an unverified `ssh-keyscan` call.

The deployment job uses its short-lived `GITHUB_TOKEN` to pull runtime images.
Separately, it creates `runtime/agent-registry/config.json` from
`GHCR_AGENT_READ_TOKEN`, assigns it to the Agent's numeric UID with mode `600`,
and mounts only that directory read-only at `/run/vulnflow-registry`. The Agent
uses it through `DOCKER_CONFIG`; it never receives the host Docker configuration
or Docker socket. Rotate the dedicated token independently of deployment access.

The API key and PostgreSQL password stay only in `runtime/.env.prod`. They are
not GitHub secrets, release-manifest values, command-line arguments, or workflow
outputs. If GHCR packages are public or the deploy user is already authenticated
securely, omit both optional GHCR settings.
outputs.

## Release flow

Expand Down
8 changes: 6 additions & 2 deletions docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,17 @@ parsing; vulnerability and artifact content are preserved.
Nginx, and production OpenAPI/Swagger default to disabled.
- The backend and agent drop Linux capabilities and use read-only root
filesystems. The non-root agent has no Docker socket.
- The Agent receives only a dedicated GHCR `read:packages` credential through a
mode-`600` Docker configuration owned by its runtime UID and mounted read-only;
it does not inherit the host Docker login.
- GitHub and host-side locks prevent overlapping production updates. Deployment
never uses `down -v`, volume deletion, or broad image pruning.

The deploy user's access to a conventional Docker daemon remains highly
privileged even when the user is not `root`. Prefer rootless Docker or a tightly
restricted command wrapper when operationally feasible. GHCR tokens, when
needed for private packages, should be read-only and dedicated to deployment.
restricted command wrapper when operationally feasible. The Agent registry token
is independent from deployment authentication and can be rotated through the
protected production environment.
Protect the `production` environment with reviewers and restrict who can change
its variables, secrets, and branch policy.

Expand Down
Loading