diff --git a/.claude/skills/add-secret/SKILL.md b/.claude/skills/add-secret/SKILL.md new file mode 100644 index 0000000..9cb097f --- /dev/null +++ b/.claude/skills/add-secret/SKILL.md @@ -0,0 +1,112 @@ +--- +name: add-secret +description: Add, rotate, or remove an environment variable or secret in a Kubernetes namespace in this repo — sealing it with kubeseal into a committed SealedSecret. Use whenever asked to add an API key, credential, token, password, or env var to prod, staging, or dev, for the pipelines/Prefect workers, the Django backend, the Next.js site, or the chatbot. +--- + +# Adding a secret + +Values are committed encrypted as SealedSecrets. Sealing is one command; the +work is in choosing the right namespaces and knowing what else has to change. +Read `CLAUDE.md` at the repository root first — it carries the conventions this +skill assumes. + +## 1. Resolve the target namespaces + +Never guess from the words "prod" or "dev". Environment naming differs per +application, and not every application has all three: + +| Application | Directories under `k8s/` | +|---|---| +| Prefect flows / pipelines | `prefect_workers/basedosdados` (prod), `prefect_workers/basedosdados-dev` (dev). **No staging exists** | +| Django backend | `website/django/{prod,staging,development}` | +| Next.js site | `website/nextjs/{production,staging,development}` | +| Chatbot | `website/chatbot/{prod,staging}` | + +If the request names an environment the application does not have, say so and +ask before inventing one — a new environment means a new namespace, Helm +release, and (for Prefect) a new work pool, not just a secret. + +Confirm which Secret the value belongs in. `make lint-secrets` prints, for every +Secret with more than one snapshot, which file is live. Existing Secrets in the +Prefect worker namespaces are `gcp-credentials` and `vault-credentials`. + +## 2. Check you can seal + +```bash +test -f .sealed-secrets-cert.pem && echo "cached cert, no cluster needed" +kubectl get ns >/dev/null 2>&1 && echo "cluster reachable" +``` + +One of those must succeed. If neither does, stop and ask the user to run +`gcloud auth login` — it cannot be done non-interactively. + +Sealing deliberately requires cluster access. Never commit +`.sealed-secrets-cert.pem` (it is gitignored), never add a CI job that seals, +and do not suggest either as a convenience: this repository is public and a +SealedSecret diff is unreviewable, so requiring cluster access is what keeps +secret authorship limited to people already trusted to apply one. `make +fetch-sealing-cert` caches it locally, which is fine — fetching still needs +cluster access. + +## 3. Seal + +Write the values to a file outside the repository so they never reach shell +history or a tracked path, and delete it afterwards. + +**A new Secret** (a `metadata.name` not yet in that directory): + +```bash +printf 'FRED_API_KEY=...\nBEA_API_KEY=...\n' > /tmp/keys.env +make seal-secret DIR=k8s/prefect_workers/basedosdados NAME=api-keys ENVFILE=/tmp/keys.env +make seal-secret DIR=k8s/prefect_workers/basedosdados-dev NAME=api-keys ENVFILE=/tmp/keys.env +rm /tmp/keys.env +``` + +Repeat per namespace. Sealing is scoped to namespace and Secret name, so one +namespace's file will not decrypt in another — this is not duplication that can +be factored out. + +**A key added to a Secret that already exists** — target the live snapshot: + +```bash +printf '...' > /tmp/value +make seal-value FILE=k8s/prefect_workers/basedosdados/secret-04_sealed.yaml \ + NAME=api-keys NAMESPACE=prefect-worker-basedosdados \ + KEY=FRED_API_KEY VALUEFILE=/tmp/value +rm /tmp/value +``` + +## 4. Verify + +```bash +make lint-secrets +pre-commit run --files k8s/ +git diff --stat +``` + +The diff should touch only the intended files. Confirm the new key appears in +`encryptedData` and that the ciphertext of untouched keys did not change. + +Never print a plaintext secret value back to the user, and never write one to a +tracked file. If a value was pasted into the conversation, do not echo it in +your summary. + +## 5. Say what remains + +Sealing writes files. It does not deploy, and for Prefect it is not even the +whole change. Hand back an explicit list: + +1. **Apply** — `kubectl apply -f `. Until then the controller + has not seen it. +2. **Wire it up, for a new Secret name in a Prefect worker namespace** — flow-run + pods only receive Secrets listed in the work pool's *base job template*, + which lives in the Prefect server database, not this repository. Add + `envFrom: - secretRef: name: ` at + `https://prefect3.basedosdados.org` → Work Pools → *pool* → Edit → Advanced, + once per pool. A new Secret is inert until this is done. Adding a key to a + Secret already listed there needs nothing. +3. **Restart** — running pods do not pick up changed Secret values. For Prefect, + the next flow run gets them; other workloads need a rollout restart. +4. **Branch and PR** — `no-commit-to-branch` blocks committing to `main` + directly. Kubernetes manifests are reviewed by hand, so the PR body should + say which namespaces are affected and what still needs applying. diff --git a/.gitignore b/.gitignore index fd6e618..bf9e815 100644 --- a/.gitignore +++ b/.gitignore @@ -8,6 +8,8 @@ ENV/ env.bak/ venv.bak/ .DS_Store +__pycache__/ +uv.lock # k8s secret.yaml @@ -15,3 +17,13 @@ secrets.yaml secret-[0-9][0-9].yaml encryption.key *.json + +# Sealed-secrets public certificate. Encrypt-only, so not confidential, but +# kept out of this public repo on purpose: needing cluster access to seal is +# what keeps "can author a secret" aligned with "can already apply one", and a +# SealedSecret diff cannot be reviewed on sight. Fetch it per machine with +# `make fetch-sealing-cert`. +.sealed-secrets-cert.pem + +# Claude Code: skills are shared, scratch worktrees are not +.claude/worktrees/ diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..1c0fcca --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,201 @@ +# CLAUDE.md + +Infrastructure for Base dos Dados: GCP resources via Terraform, cluster +workloads via Kubernetes manifests and Helm values. + +## Layout + +| Path | Contents | +|---|---| +| `terraform/` | One directory per GCP module. CI plans it, CD applies it | +| `k8s/` | One directory per namespace or application, plus its manifests | +| `utils/main.py` | Typer CLI for operational chores (sealing secrets, base64, installs) | +| `Makefile` | Aliases for the Terraform container and the secret workflow | +| `.github/workflows/` | Terraform CI/CD and Infracost | + +Everything in `k8s/` is applied by hand with `kubectl apply` or `helm upgrade`. +There is no CD pipeline for Kubernetes — the commands live in a comment at the +top of each `chart/values.yaml`. + +## One cluster, namespaces as environments + +There is a single GKE cluster, `basedosdados-dev` in `us-central1-c`, despite +the name. Production and staging are *namespaces* inside it, not separate +clusters. One consequence matters constantly: there is one sealed-secrets +controller and therefore **one sealing key for every environment**. + +Environment naming is not consistent across applications. Check the directory +before assuming: + +| Application | Environments | +|---|---| +| `k8s/prefect_workers/` | `basedosdados` (prod), `basedosdados-dev` (dev). **No staging** | +| `k8s/website/django/` | `prod`, `staging`, `development` | +| `k8s/website/nextjs/` | `production`, `staging`, `development` | +| `k8s/website/chatbot/` | `prod`, `staging` | + +## Secrets + +Secrets are committed, encrypted, as +[SealedSecrets](https://github.com/bitnami-labs/sealed-secrets). The controller +in the cluster holds the private key; the repository holds only ciphertext. +Anyone can seal a value, only the cluster can open it. + +### Numbered files are snapshots, not fragments + +This trips people up, so read it before editing anything under `k8s/`. + +Files are named `/secret-NN_sealed.yaml`, numbered from `00`. +Several files in one directory routinely declare the **same** `metadata.name`. +They are not merged. Each is a complete snapshot of that Secret at a point in +time, and the controller keeps whichever was applied last — so **the +highest-numbered file for a given Secret name is the live one**, and the lower +ones are dead history nobody has deleted. + +`k8s/website/django/prod/` holds ten sealed files. Nine declare +`api-prod-secrets`, of which only `secret-09_sealed.yaml` is live; the tenth is +an unrelated Secret. Editing `secret-04_sealed.yaml` would change nothing and +look like it should. Run `make lint-secrets` — it prints the live file for +every Secret that has more than one snapshot. + +Two ways to change a Secret, both used in the history: + +- **Add a key to the live snapshot in place** (`make seal-value`). One line + changes. This is the smaller, more reviewable diff — prefer it. +- **Seal a whole new snapshot** at the next number. Needs the plaintext of + every existing key, so it is only worth it when most values are changing. + +A genuinely *new* Secret — a different `metadata.name` — takes the next free +number, which is how `vault-credentials` came to sit at `secret-03` alongside +`gcp-credentials`. + +### Other conventions + +- Keys inside `encryptedData` are sorted alphabetically. Roughly half the + repository predates this; `make lint-secrets` reports drift as a warning + rather than an error, so old files are not a standing failure. +- No leading `---`, and no `creationTimestamp`. The `---` is not a style + preference: `pretty-format-yaml` strips it, so a file committed with one + comes back modified. The older files under `k8s/prefect_workers/` still have + theirs only because the hook has not touched them since. +- Plaintext `secret-NN.yaml` is gitignored. Prefer never creating one — the + commands below stream plaintext through a pipe and never write it to disk. + +### Adding a new Secret + +```bash +printf 'SOME_API_KEY=abc123\nOTHER_KEY=def456\n' > /tmp/new-keys.env +make seal-secret DIR=k8s/prefect_workers/basedosdados NAME=api-keys ENVFILE=/tmp/new-keys.env +rm /tmp/new-keys.env +``` + +The namespace is read from the directory's `namespace.yaml`, and the file lands +in the next free `secret-NN` slot. Repeat per environment: each namespace needs +its own file, even when the value is identical, because sealing is scoped to +namespace and Secret name by default. + +### Adding or rotating one key + +Each value in a SealedSecret is encrypted independently, so a key can be added +or replaced without knowing the plaintext of its neighbours. Point it at the +*live* snapshot: + +```bash +printf 'abc123' > /tmp/value && make seal-value \ + FILE=k8s/prefect_workers/basedosdados/secret-04_sealed.yaml \ + NAME=api-keys NAMESPACE=prefect-worker-basedosdados \ + KEY=SOME_API_KEY VALUEFILE=/tmp/value && rm /tmp/value +``` + +This rewrites one line, which is what a well-scoped secret commit looks like — +see `6be5d07`. `VALUE=` works too but lands in shell history; prefer +`VALUEFILE=`. + +### Sealing requires cluster access, on purpose + +`kubeseal` encrypts with the controller's public certificate, which it fetches +from the current `kubectl` context — so sealing needs a live `gcloud auth +login`. That is a deliberate control, not friction to be engineered away. + +The certificate is encrypt-only and therefore not confidential; Bitnami +documents publishing it as safe. The reason it stays out of this repository is +different. **This repository is public, and a SealedSecret diff is unreviewable +by construction** — ciphertext in, ciphertext out. While producing valid +ciphertext requires cluster access, the set of people who can author a secret +is exactly the set already trusted to apply one. Committing the certificate +would break that correspondence and let anyone open a PR sealing an arbitrary +value into, say, `vault-credentials`, with nothing visible in review. + +So: do not commit `.sealed-secrets-cert.pem`, and do not add a CI job that +seals. Review a secret PR by who wrote it and what they say it contains, then +confirm against the cluster after applying. + +If you would rather not re-authenticate for every seal, cache the certificate +locally — it is gitignored, and fetching it still requires cluster access: + +```bash +make fetch-sealing-cert # writes .sealed-secrets-cert.pem, git-ignored +``` + +When that file exists, `seal-secret` and `seal-value` use it and skip the +cluster. Delete it and re-fetch if the controller's key is rotated. + +### Applying + +Sealing writes a file; it does not deploy. The controller only sees it after: + +```bash +kubectl apply -f k8s/prefect_workers/basedosdados/secret-04_sealed.yaml +``` + +Running pods do not pick up changed Secret values on their own. Restart the +consumer, or let the next flow run pick it up. + +## Prefect + +`k8s/prefect3/` is the Prefect *server* — API, UI, and its Cloud SQL connection. +It does not run flows. + +`k8s/prefect_workers/` holds the two workers that do. Each polls a work pool of +the same name and launches one Kubernetes Job per flow run, in its own +namespace. `basedosdados-dev` runs flows from PR branches without schedules; +`basedosdados` runs scheduled production flows from `main`. The +[pipelines](https://github.com/basedosdados/pipelines) repository targets them +by pool name in `.github/scripts/deploy_flows.py`. + +**Flow-run pods do not automatically see the Secrets in their namespace.** The +`envFrom` list lives in the work pool's *base job template*, which is stored in +the Prefect server's database and edited in the UI at +`https://prefect3.basedosdados.org` under Work Pools → *pool* → Edit → Advanced. +A newly sealed Secret is inert until its name is added there, once per pool. Say +so explicitly when handing off a new Secret — the manifest alone is not the +whole change. + +Existing Secrets, both workers: `gcp-credentials` (`BASEDOSDADOS_CONFIG`, +`BASEDOSDADOS_CREDENTIALS_PROD`, `BASEDOSDADOS_CREDENTIALS_STAGING`, +`DBT_SERVICE_ACCOUNT`) and `vault-credentials` (`VAULT_ADDRESS`, +`VAULT_TOKEN`). The `_PROD` / `_STAGING` suffix names a BigQuery dataset tier, +not a deployment environment — that distinction has caused confusion before. + +`k8s/prefect_workers/basedosdados/` carries both `secret-01` and `secret-02` as +`gcp-credentials`; `secret-02` is the live one. + +## Conventions + +- Commits follow `type(scope): description`; the history is mostly Portuguese + for prose and English for the summary line. Either is accepted. +- `pre-commit` reformats YAML to two-space indent and trims whitespace. Run + `pre-commit run --files ` before committing generated manifests. +- `no-commit-to-branch` blocks direct commits to `main`. Branch, then PR. +- Terraform changes are planned in CI and shown on the PR. Kubernetes manifests + are reviewed by hand — describe what you applied and when. + +## Verification + +`make lint-secrets` reads every sealed manifest, decrypting nothing and needing +no cluster. It fails on the mistakes that break a deploy — a missing `template` +stanza, a namespace that disagrees with the directory's `namespace.yaml`, a +`metadata.name` that disagrees with the template's, an empty `encryptedData`. +Style drift (unsorted keys, missing `---`) is reported as a warning; pass +`--strict` to fail on it too. Its `note:` lines name the live snapshot for +every Secret that has more than one. diff --git a/Makefile b/Makefile index 4f4aefb..0b84e66 100644 --- a/Makefile +++ b/Makefile @@ -25,6 +25,47 @@ update-dev: . .venv/bin/activate; \ poetry update; +.PHONY: fetch-sealing-cert lint-secrets seal-secret seal-value + +# Python entrypoint for utils/main.py. Uses the project venv when its +# dependencies are actually installed (`make create-dev`), otherwise falls back +# to uv, which resolves typer on the fly. Probing for the venv directory is not +# enough -- an empty .venv is a common leftover. +UTILS := $(shell .venv/bin/python -c "import typer" 2>/dev/null \ + && echo ".venv/bin/python utils/main.py" \ + || echo "uv run --quiet --with typer python utils/main.py") + +# Fetch the sealed-secrets public certificate once, so later sealing needs no +# cluster access. Requires a live `gcloud auth login`. +fetch-sealing-cert: + $(UTILS) fetch-sealing-cert + +# Report which snapshot is live per Secret, and fail on structural mistakes. +lint-secrets: + $(UTILS) lint-secrets + +# Create a new SealedSecret. The namespace is read from the directory's +# namespace.yaml; the file lands in the next free secret-NN slot. +# make seal-secret DIR=k8s/prefect_workers/basedosdados NAME=api-keys ENVFILE=/tmp/keys.env +seal-secret: + @test -n "$(DIR)" || (echo "set DIR=" && exit 1) + @test -n "$(NAME)" || (echo "set NAME=" && exit 1) + @test -n "$(ENVFILE)" || (echo "set ENVFILE=" && exit 1) + $(UTILS) seal-secret --directory $(DIR) --name $(NAME) --from-env-file $(ENVFILE) \ + $(if $(INDEX),--index $(INDEX),) + +# Add or rotate a single key inside an existing SealedSecret, without needing +# the plaintext of the other keys. Prefer VALUEFILE= over VALUE= -- the latter +# lands in shell history. +# make seal-value FILE=... NAME=api-keys NAMESPACE=... KEY=FRED_API_KEY VALUEFILE=/tmp/fred +seal-value: + @test -n "$(FILE)" || (echo "set FILE=" && exit 1) + @test -n "$(NAME)" || (echo "set NAME=" && exit 1) + @test -n "$(NAMESPACE)" || (echo "set NAMESPACE=" && exit 1) + @test -n "$(KEY)" || (echo "set KEY=" && exit 1) + $(UTILS) seal-value --into $(FILE) --name $(NAME) --namespace $(NAMESPACE) --key $(KEY) \ + $(if $(VALUEFILE),--value-file $(VALUEFILE),--value '$(VALUE)') + .PHONY: docker-clean docker-down docker-force docker-logs docker-start docker-stop docker-up docker-clean: diff --git a/k8s/prefect_workers/basedosdados-dev/secret-03_sealed.yaml b/k8s/prefect_workers/basedosdados-dev/secret-03_sealed.yaml new file mode 100644 index 0000000..a610886 --- /dev/null +++ b/k8s/prefect_workers/basedosdados-dev/secret-03_sealed.yaml @@ -0,0 +1,14 @@ +apiVersion: bitnami.com/v1alpha1 +kind: SealedSecret +metadata: + name: api-keys + namespace: prefect-worker-basedosdados-dev +spec: + encryptedData: + BEA_API_KEY: AgBt/gIe7dpj8G90OQJpHLVvx4P3aW+PrKAoyPt05xkfagS459AcQ9/w0sXOlv6G4ZvJb2V3e/hs8/Y7DHaytX8ya6avk3az7ZzDN3BoQelErHLk4n7OcYwwzN3PT7PdOkt5i5hOo+clOxIA9pqGIqcK9V27sIl08GOtx+v3DTiL1XIOmKYBBqsqNozkrTsqfskH6UsX+XpFNmfIBREm/x5ExaE4vqjdG+22sqyVSQTmO4EfDyjGa4uJRcBib7R7rt6vF+0ucYzh4lrW20IYUVPoRVPmNRKN4XkmTuppM8NB5y3iFeRqslhgKmjAy0pdZub//IbaHqJsGmks+xVFhpD6Cvib4zjwnW5CW9EkhDz+6KJ6jGt3l5+8sFZOiRx9M+TMVzbsyXjxB3zTor6aEz6l7MS2dpRxL9iFpDGwdFAnlQEdW+95BIn3FYlygW/DtnSMe97a+R97eKPzysmEqOghrZVxXIOclnzF/70MH/ttIdeKh8ICW5ovfspZPi6GCtUYlI5XXO0Ox08A8i4w/FJvAcYfrnh2XFsf6lGRt+jD1AO9WNbyawgwR9VyrL0g+jYvuR6PAwYipcNVIGkkWN/Z59Om9vFhYg0mNC66+X0T/AHFnp9nyaAO23w+5/2Zek+PAktuY/yBuA4fIXm2hCNzX4WU5zA5bZXuW2mzEdVMaWoC4jn6hr/i8//GY/9LcPCF9kyDdzmrVXStDXLvdHW4eufx/RoeZS9Ls/G/qHtvuWGuBLc= + FRED_API_KEY: AgCoyOsXzttizZlQtnBDdJKVIyJDM6/hHZoiM7SKwzqMpQS8HWRZBm2DNvR0CNiubpgzRL2yzCTBqag8XDhB+7fntfZyh8XeIW5YPgh9EMs1GbxinNygcoWGPk++rlUn87uteKioQfCzMXdzu3OX5ER8NqdE/Wv3b4dRg9JHgSQdwaSR3tLld7eBdZF1HEnzuLlUk5OPW/hlRxktafsnn2RW5ah5lru8DEz3xgnZQytCShFlmUB0KL4RHUi3/RYcngugpsvuR5jlPp/pOEl7nTFrJ6TXz2bdoU4jTw/kAH3xpoavexpJn3l5EuyHWiwtHFtdhYSq/1QwyLoQT6F0vcjr8UrN0MsrokEqUYbtME97KO9TSXX2AQ4sExlhr+dx3DGQLf1nEpHzhIm02P0sA1XhcXnfV4PGwnLWgOUcRnQOcj6LX6RSDHV/i3A75dLhPS70XwbIVIDd/pEN6z/tU0OCoEneyfA0q4ZxaWVkmzyZ6StOK3fcaq609X3GWAVKg7bohCM8gPWfOGWIlUwxuBwKybw2y7nmfhMc0TTc3nWkXhymtWRU2iOYtGvTNBsWFNDNay1OZQDxfBpsIy7AeniOkZaglsqwIZqTvSvKn29rM8NXSiaBeP/M5rt1gVtWo90bUnMpbrNf81pCzXbRbTbEGlkchzdQma0L57wASgFVYYw5dTZEh2vHRdtY9fGx12akjeju1Kya9yA8Ovq1wYM8uh+hNsvbB9paGyV9ps40bw== + template: + metadata: + name: api-keys + namespace: prefect-worker-basedosdados-dev + type: Opaque diff --git a/k8s/prefect_workers/basedosdados/secret-04_sealed.yaml b/k8s/prefect_workers/basedosdados/secret-04_sealed.yaml new file mode 100644 index 0000000..c3b0458 --- /dev/null +++ b/k8s/prefect_workers/basedosdados/secret-04_sealed.yaml @@ -0,0 +1,14 @@ +apiVersion: bitnami.com/v1alpha1 +kind: SealedSecret +metadata: + name: api-keys + namespace: prefect-worker-basedosdados +spec: + encryptedData: + BEA_API_KEY: AgCnK99SwtaEwW4kizlrSaJATYh76z9fv5LfH0WP57aHNJR0OKMIzkNwEayQV6q9qn+zJEioIY8IUN4jfoeGwdTO84Q//+1uufw3wlqxRFVY1iLm8ZuIQXGaU5Q3AhHD9946qMs2J5/pMZoAn/v44Hewjsiey6YHH3bSAl6mxggceYVIGIgs8q5CJBflM9D7WCKfRG5H3aqWWXSCZ51VzptvRwZVHPEDzJQyBFyumNIxJEC1GaZD8BMJ352YrbkuETKAMK2ALRV/tZjszQ4OfSP1VbZi93wUsbdb6NsoDIcni4bjdU923NFPg/80EvsQbL0G/Bdcw0VR6Y+sMmi8wr9B2jSgIRAVcUTu42nVPYnde506s1oeV3aw9PAglnhpae9k38znHwAkaWok5XMceCASUfiKpj0dBh3qPkhsHKmQaAeYepxy0QRkH+JHYXoh87WvyQfIwOP6xi/n6wsOpYtEGAM4TM2jz/QrjBPAmjXCa6hDOBRWyf9fTAQe8zfw+6gu4T+APD2Y3yf29fWdfY18QN8+3LzKTeNfj0z83xTTv1rv6JKW1oYH4lr+HcNEZvFzTGvbOnIBhFaDgUsxRZFv3ZSOfZE3VITFOHTYS5yIm5o9vuZL9907imQyq2/gJXgjLz/jb7vLgPrAlir5CLNVVY3hM+paJtUisyUESv/5FwLIM0HdKrsj/dKW4wIwyf53pe01ZPJg0Elv8Sev4PSMQYn4j06CcbxMN8DdDDbLby36T2Q= + FRED_API_KEY: AgAUUOesBaly6xvFX8aVYasUao63e3BcaNRc8FTWXBjCmW1itJ30RrNrFPAYp1Ge8a1B7ZJ7TbwgnxQq7nxJv3WG3VsOpTNx7IfZ5Q4sxtu5tJAjHjteATC5Cfl0sN2lBzkc/AmjJQoC1hJ+5XOOALobavERGGJ20KdtBUkQg7RCqcpllVFxonnTaEbndxeHuVpaT0wgIeKcP7hixtnyzA1oDyuW9+xFw0UV5ZWoi6uuhavvxPxKjBrPem+kRUbvLiH547yrPVATfwaNnjDK/Gf1Z6UcLdhBKzZPza6IebaX0oCa3sugKhJmvtkGiaDnxexQ7gImx1imYm9QlRYfGu4cRL6DDuJoCR8yBEUnP2+dwwr390xtK6pxxtjkiwIDlkzbE24EFw4E7rSkKOwxoP0TDanUjpldyKZ5hlhiOLB8vkoQWgTNkx+Gyl+rr8/rUZ5e+QaD0j3z8CQt7z6CFpFd3ivX5vFrByBoy0pBEy+A38htZNxg+owzD9EwQFfqLAPGuRoA57AuyocCpGn9Mb8KADgxv2XzigJx12uc/4z46nltdwSL65IYm4kSnLNs3gDRihhYDXl6qyON+2i0a3vOnm1EaT3n6el4KtU1o2ezvsTZo1G3vhDXN9MWTl6M6AGdfWTjj1caaf3dRcJ3tf4zrsv5ej6Oej6/461MGA2grh85Q6Tw0h942bcc0+aBepwGzcnXGR73nIvoAKDcZdqO1JjqRfyu5JuWJj02rn+jtA== + template: + metadata: + name: api-keys + namespace: prefect-worker-basedosdados + type: Opaque diff --git a/utils/main.py b/utils/main.py index da95437..7ca188d 100644 --- a/utils/main.py +++ b/utils/main.py @@ -1,6 +1,7 @@ from functools import partial +from pathlib import Path import subprocess -from typing import Callable +from typing import Callable, Dict, List, Optional import base64 import typer import random @@ -8,6 +9,14 @@ app = typer.Typer() +# Certificado público do controlador sealed-secrets, obtido sob demanda por +# `fetch-sealing-cert`. Deliberadamente fora do versionamento (ver .gitignore): +# este repositório é público, e o diff de um SealedSecret é ilegível por +# construção. Enquanto o certificado exigir acesso ao cluster, quem consegue +# cifrar um segredo é exatamente quem já tem acesso para aplicá-lo. Publicá-lo +# quebraria essa correspondência. +SEALING_CERT = Path(".sealed-secrets-cert.pem") + def command_exists(command: str) -> bool: """ @@ -36,6 +45,352 @@ def echo_and_run(command: str, stdout_callback: Callable = partial(print, end='' return return_code +def fail(message: str): + """ + Aborts with an error message + """ + typer.echo(f"error: {message}", err=True) + raise typer.Exit(1) + + +def run_quietly(command: List[str], stdin_data: Optional[str] = None) -> str: + """ + Runs a command without echoing it, returning stdout. Used wherever the + command or its input carries secret material, so nothing leaks to the + terminal or to shell history. + """ + result = subprocess.run( + command, input=stdin_data, capture_output=True, text=True) + if result.returncode: + fail(f"{command[0]} failed: {result.stderr.strip()}") + return result.stdout + + +def kubeseal_flags(controller_name: str, controller_namespace: str) -> List[str]: + """ + Builds the flags that tell kubeseal which public key to encrypt with. + Prefers the versioned certificate (offline); falls back to fetching it + from the controller in the current kubectl context. + """ + if SEALING_CERT.exists(): + return ["--cert", str(SEALING_CERT)] + return [ + "--controller-name", controller_name, + "--controller-namespace", controller_namespace, + ] + + +def collect_values(from_literal: Optional[List[str]], + from_env_file: Optional[str]) -> Dict[str, str]: + """ + Gathers KEY=VALUE pairs from an env file and/or repeated --from-literal + flags. The env file is preferred: values passed on the command line end up + in shell history. + """ + values: Dict[str, str] = {} + sources = [] + if from_env_file: + path = Path(from_env_file) + if not path.is_file(): + fail(f"env file not found: {from_env_file}") + sources = [ + line.strip() for line in path.read_text().splitlines() + if line.strip() and not line.strip().startswith("#") + ] + sources += list(from_literal or []) + for entry in sources: + key, separator, value = entry.partition("=") + if not separator: + fail(f"expected KEY=VALUE, got: {key}") + values[key.strip()] = value.strip().strip('"').strip("'") + if not values: + fail("no values given -- use --from-env-file or --from-literal") + return values + + +def namespace_of(directory: Path) -> str: + """ + Reads the namespace from the namespace.yaml sitting next to the secrets, + so the namespace never has to be retyped (and so it cannot drift). + """ + manifest = directory / "namespace.yaml" + if not manifest.is_file(): + fail(f"no namespace.yaml in {directory} -- pass --namespace explicitly") + for line in manifest.read_text().splitlines(): + if line.strip().startswith("name:"): + return line.split(":", 1)[1].strip() + fail(f"could not read metadata.name from {manifest}") + + +def next_index(directory: Path) -> str: + """ + Returns the slot after the highest one in use. Deliberately not the lowest + free slot: a higher number means a newer snapshot, and several directories + start at 01, so filling a gap at 00 would read as the oldest file. + """ + used = { + int(path.name[len("secret-"):][:2]) + for path in directory.glob("secret-[0-9][0-9]_sealed.yaml") + } + return f"{(max(used) + 1) if used else 0:02d}" + + +def secret_manifest(name: str, namespace: str, values: Dict[str, str]) -> str: + """ + Renders a plain Secret manifest, base64-encoding each value. Kept in memory + and piped straight into kubeseal -- the plaintext never touches disk. + """ + lines = [ + "apiVersion: v1", + "kind: Secret", + "metadata:", + f" name: {name}", + f" namespace: {namespace}", + "type: Opaque", + "data:", + ] + for key in sorted(values): + lines.append(f" {key}: {base64.b64encode(values[key].encode()).decode()}") + return "\n".join(lines) + "\n" + + +def tidy(sealed: str) -> str: + """ + Strips the null creationTimestamp that kubeseal emits. Deliberately does + not add a leading '---': the pretty-format-yaml pre-commit hook removes it + again, so adding one only produces a spurious diff at commit time. + """ + kept = [ + line for line in sealed.splitlines() + if line.strip() != "creationTimestamp: null" + ] + return "\n".join(kept) + "\n" + + +def splice_key(path: Path, key: str, encrypted: str): + """ + Inserts (or replaces) a single key inside an existing SealedSecret's + encryptedData block, keeping the keys in alphabetical order. Every value in + a SealedSecret is encrypted independently, so one key can be added without + knowing the plaintext of its neighbours. + """ + lines = path.read_text().splitlines() + starts = [i for i, line in enumerate(lines) if line.rstrip() == " encryptedData:"] + if not starts: + fail(f"no encryptedData block in {path}") + start = starts[0] + end = start + 1 + while end < len(lines) and lines[end].startswith(" "): + end += 1 + block = [line for line in lines[start + 1:end] + if not line.startswith(f" {key}:")] + block.append(f" {key}: {encrypted}") + block.sort(key=lambda line: line.split(":", 1)[0].strip()) + path.write_text("\n".join(lines[:start + 1] + block + lines[end:]) + "\n") + + +@app.command() +def fetch_sealing_cert( + controller_name: str = "sealed-secrets-controller", + controller_namespace: str = "kube-system", + output: str = str(SEALING_CERT), +): + """ + Fetches the sealed-secrets public certificate from the cluster and writes it + to disk. Requires cluster access, but only once: the certificate is public + and versioned, so subsequent sealing runs offline. + """ + if not command_exists("kubeseal"): + fail("kubeseal not found -- install it with `brew install kubeseal`") + certificate = run_quietly([ + "kubeseal", "--fetch-cert", + "--controller-name", controller_name, + "--controller-namespace", controller_namespace, + ]) + destination = Path(output) + destination.parent.mkdir(parents=True, exist_ok=True) + destination.write_text(certificate) + typer.echo(f"wrote {destination}") + + +@app.command() +def seal_secret( + directory: str = typer.Option(..., "--directory", "-d", + help="Namespace directory under k8s/"), + name: str = typer.Option(..., "--name", "-n", + help="metadata.name of the Secret"), + namespace: str = typer.Option(None, help="Defaults to the directory's namespace.yaml"), + from_env_file: str = typer.Option(None, help="File of KEY=VALUE lines"), + from_literal: List[str] = typer.Option(None, help="KEY=VALUE, repeatable"), + index: str = typer.Option(None, help="Two-digit slot; defaults to the next free one"), + controller_name: str = "sealed-secrets-controller", + controller_namespace: str = "kube-system", +): + """ + Creates a new SealedSecret at /secret-NN_sealed.yaml + """ + if not command_exists("kubeseal"): + fail("kubeseal not found -- install it with `brew install kubeseal`") + target = Path(directory) + if not target.is_dir(): + fail(f"not a directory: {directory}") + namespace = namespace or namespace_of(target) + values = collect_values(from_literal, from_env_file) + sealed = run_quietly( + ["kubeseal", "--format", "yaml"] + kubeseal_flags(controller_name, controller_namespace), + stdin_data=secret_manifest(name, namespace, values), + ) + destination = target / f"secret-{index or next_index(target)}_sealed.yaml" + destination.write_text(tidy(sealed)) + typer.echo(f"wrote {destination} ({', '.join(sorted(values))} -> {namespace}/{name})") + + +@app.command() +def seal_value( + name: str = typer.Option(..., "--name", "-n", help="metadata.name of the Secret"), + namespace: str = typer.Option(..., "--namespace", help="Namespace of the Secret"), + key: str = typer.Option(..., "--key", "-k", help="Key to add or replace"), + value: str = typer.Option(None, help="Value; omit to read from --value-file"), + value_file: str = typer.Option(None, help="File whose contents are the value"), + into: str = typer.Option(None, help="SealedSecret file to splice the key into"), + controller_name: str = "sealed-secrets-controller", + controller_namespace: str = "kube-system", +): + """ + Seals a single value. Adds or replaces one key in an existing SealedSecret + without needing the plaintext of the other keys. + """ + if not command_exists("kubeseal"): + fail("kubeseal not found -- install it with `brew install kubeseal`") + if (value is None) == (value_file is None): + fail("pass exactly one of --value or --value-file") + plaintext = value if value is not None else Path(value_file).read_text() + encrypted = run_quietly( + ["kubeseal", "--raw", "--name", name, "--namespace", namespace, + "--from-file", "/dev/stdin"] + + kubeseal_flags(controller_name, controller_namespace), + stdin_data=plaintext, + ).strip() + if not into: + typer.echo(encrypted) + return + destination = Path(into) + if not destination.is_file(): + fail(f"not a file: {into}") + splice_key(destination, key, encrypted) + typer.echo(f"set {key} in {destination}") + + +def read_sealed(path: Path) -> Dict[str, object]: + """ + Pulls the fields worth checking out of a sealed manifest, without a YAML + parser: the Secret's name, the namespaces it mentions, and its key list. + """ + lines = path.read_text().splitlines() + names = [line.split(":", 1)[1].strip() for line in lines + if line.strip().startswith("name:")] + namespaces = {line.split(":", 1)[1].strip() for line in lines + if line.strip().startswith("namespace:")} + keys: List[str] = [] + starts = [i for i, line in enumerate(lines) if line.rstrip() == " encryptedData:"] + if starts: + cursor = starts[0] + 1 + while cursor < len(lines) and lines[cursor].startswith(" "): + keys.append(lines[cursor].split(":", 1)[0].strip()) + cursor += 1 + return { + "name": names[0] if names else None, + "names": names, + "namespaces": namespaces, + "keys": keys, + "has_template": any(line.strip() == "template:" for line in lines), + "leads_with_marker": bool(lines) and lines[0].strip() == "---", + "index": int(path.name[len("secret-"):][:2]), + } + + +@app.command() +def lint_secrets(root: str = "k8s", strict: bool = False): + """ + Checks sealed manifests for the mistakes that actually break a deploy, and + reports which snapshot is live where a Secret has several. Decrypts nothing + and needs no cluster access. --strict also fails on style drift. + """ + errors: List[str] = [] + warnings: List[str] = [] + notes: List[str] = [] + directories: Dict[Path, List[Path]] = {} + for path in sorted(Path(root).rglob("secret-[0-9][0-9]_sealed.yaml")): + directories.setdefault(path.parent, []).append(path) + + for directory, paths in sorted(directories.items()): + declared = None + namespace_manifest = directory / "namespace.yaml" + if namespace_manifest.is_file(): + for line in namespace_manifest.read_text().splitlines(): + if line.strip().startswith("name:"): + declared = line.split(":", 1)[1].strip() + break + + snapshots: Dict[str, List[Path]] = {} + for path in paths: + manifest = read_sealed(path) + + if not manifest["keys"]: + errors.append(f"{path}: no encryptedData block, or it is empty") + if not manifest["has_template"]: + errors.append( + f"{path}: missing template stanza -- the created Secret " + f"would lose its name and type") + if len(set(manifest["names"])) > 1: + errors.append( + f"{path}: metadata.name and template name disagree " + f"-- {sorted(set(manifest['names']))}") + if declared and manifest["namespaces"] - {declared}: + errors.append( + f"{path}: targets namespace " + f"{sorted(manifest['namespaces'] - {declared})}, but " + f"namespace.yaml declares {declared}") + + if manifest["keys"] != sorted(manifest["keys"]): + warnings.append(f"{path}: encryptedData keys are not sorted") + if manifest["leads_with_marker"]: + warnings.append( + f"{path}: leading '---' will be stripped by " + f"pretty-format-yaml on the next commit that touches it") + + if manifest["name"]: + snapshots.setdefault(manifest["name"], []).append(path) + + # Several files may declare one Secret. They are successive snapshots, + # not a merge: the controller keeps whichever was applied last, so the + # highest-numbered file is the live one and the rest are history. + for name, versions in sorted(snapshots.items()): + if len(versions) > 1: + live = max(versions, key=lambda path: read_sealed(path)["index"]) + superseded = ", ".join( + path.name for path in sorted(versions) if path != live) + notes.append( + f"{directory}: {name} -- live is {live.name}; " + f"superseded: {superseded}") + + total = sum(len(paths) for paths in directories.values()) + for note in notes: + typer.echo(f"note: {note}") + for warning in warnings: + typer.echo(f"warning: {warning}", err=True) + for error in errors: + typer.echo(f"error: {error}", err=True) + + if errors: + fail(f"{len(errors)} error(s) across {total} manifests") + if warnings and strict: + fail(f"{len(warnings)} style warning(s) across {total} manifests") + typer.echo( + f"ok -- {total} manifests in {len(directories)} directories, " + f"{len(warnings)} style warning(s)") + + @app.command() def decode_base64(data: str): """