Repository navigation
chore(dev): Compose stack with PostgreSQL 17 and SeaweedFS, database roles and Makefile #23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # Local development only. Every value below is a dummy; never reuse in production. | ||
| # `make up` copies this file to .env (ignored by git), which Compose reads: the keys below are the one place the | ||
| # development file-store keys and database password are written down for Compose (the db/init SQL repeats the password). | ||
| COMPOSE_PROJECT_NAME=grimoire | ||
| GRIMOIRE_DEV_DB_PASSWORD=grimoire-dev | ||
| GRIMOIRE_DEV_S3_ACCESS_KEY=grimoire-dev | ||
| GRIMOIRE_DEV_S3_SECRET_KEY=grimoire-dev-secret | ||
| GRIMOIRE_DATABASE_URL=postgresql://grimoire_app:grimoire-dev@127.0.0.1:55432/grimoire | ||
| GRIMOIRE_DATABASE_MIGRATION_URL=postgresql://grimoire_migrator:grimoire-dev@127.0.0.1:55432/grimoire | ||
| GRIMOIRE_S3_ENDPOINT=http://127.0.0.1:8333 | ||
| GRIMOIRE_S3_BUCKET=grimoire | ||
| GRIMOIRE_S3_ACCESS_KEY=grimoire-dev | ||
| GRIMOIRE_S3_SECRET_KEY=grimoire-dev-secret | ||
| # GRIMOIRE_DEV_TOKEN_SECRET is not written here on purpose: `make up` generates a random one into .env on first use (a fixed value would be a shared password). | ||
|
omercnet marked this conversation as resolved.
Resolved
|
||
| # Empty is safe for local development; a non-local server requires a comma-separated list of lowercase domains. | ||
| GRIMOIRE_ALLOWED_EMAIL_DOMAINS= | ||
| # Empty selects the default Descope API endpoint; use an HTTP(S) URL only for a custom or local key server. | ||
| # Maximum accepted issued-token lifetime: 24 hours; the loader accepts 1 to 86400 seconds. | ||
| GRIMOIRE_DESCOPE_MAX_TOKEN_LIFETIME_SECONDS=86400 | ||
| GRIMOIRE_DESCOPE_BASE_URL= | ||
| # Always empty in this example: this is a secret, and no real or realistic key belongs in the repository. | ||
| GRIMOIRE_DESCOPE_MANAGEMENT_KEY= | ||
| # Re-check collaboration access every 60 seconds; the loader accepts an integer from 1 to 600. | ||
| GRIMOIRE_COLLAB_RECHECK_SECONDS=60 | ||
| # Unauthenticated sockets are closed and released after 5 seconds; the loader accepts 1 to 60 seconds. | ||
| GRIMOIRE_COLLAB_PREAUTH_TIMEOUT_SECONDS=5 | ||
| # Authenticated resources per user: 20 sockets, 50 document sessions, and 2 MiB of updates per second across pages. | ||
| GRIMOIRE_COLLAB_MAX_SOCKETS_PER_USER=20 | ||
| GRIMOIRE_COLLAB_MAX_DOCUMENTS_PER_USER=50 | ||
| GRIMOIRE_COLLAB_BYTES_PER_SECOND_PER_USER=2097152 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| # COMPOSE_FILE (for example compose.yaml:compose.offline.yaml) is honoured; the project name is fixed. | ||
| COMPOSE := docker compose -p grimoire | ||
|
|
||
| .PHONY: db-up db-down down down-volumes test | ||
|
|
||
| # Compose reads .env for the development passwords and keys; .env.example is the single place they are written down. | ||
| # Hand-written: Compose has no "copy a default env file" feature. | ||
| .env: | ||
| cp .env.example .env | ||
| chmod 600 .env | ||
|
|
||
| # Development tokens carry a per-checkout secret (GRIMOIRE_DEV_TOKEN_SECRET): generated on first use, never a fixed default, kept in the ignored .env | ||
| # (mode 600). 32 bytes from the kernel's CSPRNG (/dev/urandom), hex encoded (64 characters). An existing value is never rewritten. | ||
| # Every target that starts the app or reads its environment depends on this one (compose-settings.test.ts checks `up` reaches it). | ||
|
omercnet marked this conversation as resolved.
Resolved
|
||
| .PHONY: dev-secret | ||
| dev-secret: .env | ||
| @chmod 600 .env | ||
| @grep -q '^GRIMOIRE_DEV_TOKEN_SECRET=.' .env || { [ -z "$$(tail -c1 .env)" ] || echo >> .env; echo "GRIMOIRE_DEV_TOKEN_SECRET=$$(head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n')" >> .env; echo "generated GRIMOIRE_DEV_TOKEN_SECRET in .env"; } | ||
|
|
||
| # NOTE: database initialisation scripts (db/init) run only on an EMPTY volume. A database volume created before | ||
| # the roles or passwords changed must be reset with `make reset` (or `make down-volumes`, which deletes the data) before the next `make up` or `make db-up`. | ||
|
omercnet marked this conversation as resolved.
Resolved
|
||
| db-up: .env dev-secret | ||
| $(COMPOSE) up -d --wait --wait-timeout 120 postgres seaweedfs | ||
|
|
||
|
|
||
| # Stops and removes the containers and the network; the data volumes stay, so `make up` brings the same data back. | ||
| down: | ||
| $(COMPOSE) --profile tools down --remove-orphans | ||
|
|
||
| db-down: down | ||
|
|
||
| # The only targets that delete data (database and file store volumes). | ||
| down-volumes: | ||
| $(COMPOSE) --profile tools down -v --remove-orphans | ||
|
|
||
| test: | ||
| bun run test | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| name: grimoire | ||
|
|
||
| services: | ||
| postgres: | ||
| image: postgres:17@sha256:b60e90042aa6b8dbecd58e224b3e36114dbf8ad754465264c2476ad58a86e45c | ||
| environment: | ||
| POSTGRES_DB: grimoire | ||
| POSTGRES_USER: grimoire_admin | ||
| POSTGRES_PASSWORD: ${GRIMOIRE_DEV_DB_PASSWORD} # development only; value in .env (copied from .env.example) | ||
| GRIMOIRE_DEV_DB_PASSWORD: ${GRIMOIRE_DEV_DB_PASSWORD} # read by db/init/01-roles.sh | ||
| ports: | ||
| - "127.0.0.1:55432:5432" | ||
| volumes: | ||
| - grimoire-pgdata:/var/lib/postgresql/data | ||
| - ./db/init:/docker-entrypoint-initdb.d:ro | ||
| healthcheck: | ||
| test: ["CMD-SHELL", "pg_isready -U grimoire_admin -d grimoire"] | ||
|
omercnet marked this conversation as resolved.
Resolved
|
||
| interval: 2s | ||
| timeout: 3s | ||
| retries: 30 | ||
| start_period: 5s | ||
|
|
||
| seaweedfs: | ||
| image: chrislusf/seaweedfs:4.54@sha256:4b42c212543a0acd4c0e675215ecdcaf62a186ef98def76c7e12701871222d15 | ||
| command: ["mini", "-dir=/data", "-s3", "-ip.bind=0.0.0.0"] | ||
| environment: | ||
| AWS_ACCESS_KEY_ID: ${GRIMOIRE_DEV_S3_ACCESS_KEY} # development only; value in .env | ||
| AWS_SECRET_ACCESS_KEY: ${GRIMOIRE_DEV_S3_SECRET_KEY} # development only; value in .env | ||
| ports: | ||
| - "127.0.0.1:8333:8333" | ||
| volumes: | ||
| - grimoire-seaweed:/data | ||
| healthcheck: | ||
| # Signed ListBuckets must return HTTP 200 with a ListAllMyBucketsResult body: | ||
| # proves the S3 gateway is serving and the filer/master behind it answer. | ||
| test: ["CMD-SHELL", "curl -sf --aws-sigv4 aws:amz:us-east-1:s3 --user \"$$AWS_ACCESS_KEY_ID:$$AWS_SECRET_ACCESS_KEY\" http://127.0.0.1:8333/ | grep -q ListAllMyBucketsResult"] | ||
| interval: 2s | ||
| timeout: 3s | ||
| retries: 30 | ||
| start_period: 3s | ||
|
|
||
| volumes: | ||
| grimoire-pgdata: | ||
| grimoire-seaweed: | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| #!/bin/sh | ||
| # Hand-written because the postgres image runs *.sql init files without variables: this passes the shared dev password to psql. | ||
| set -eu | ||
| psql -v ON_ERROR_STOP=1 -v pw="$GRIMOIRE_DEV_DB_PASSWORD" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -f /docker-entrypoint-initdb.d/roles.psql |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| -- Local development only. Run once by the postgres image's init step (after 01-roles.sh), as the container superuser, against database grimoire. | ||
| -- grimoire_migrator has only CONNECT on the database and cannot create extensions; migration 0004 repeats `create extension if not exists citext`, which is a no-op once this ran. | ||
| -- Both are trusted extensions. On a managed service (Cloud SQL) the administrator is not a superuser: install them there with a role that may create extensions | ||
| -- (cloudsqlsuperuser) before the first `grimoire migrate`. | ||
| CREATE EXTENSION IF NOT EXISTS citext; | ||
| CREATE EXTENSION IF NOT EXISTS pg_trgm; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| -- Local development only. Run once by 01-roles.sh (psql -v pw=...), as the container superuser grimoire_admin, against database grimoire. | ||
| -- The password is the GRIMOIRE_DEV_DB_PASSWORD value, passed as the psql variable :pw. | ||
| CREATE ROLE grimoire_migrator LOGIN PASSWORD :'pw'; | ||
| CREATE ROLE grimoire_app LOGIN PASSWORD :'pw'; | ||
|
|
||
| REVOKE ALL ON DATABASE grimoire FROM PUBLIC; | ||
| GRANT CONNECT ON DATABASE grimoire TO grimoire_migrator, grimoire_app; | ||
|
|
||
| -- The migration role owns the schema and may change it. | ||
| ALTER SCHEMA public OWNER TO grimoire_migrator; | ||
| REVOKE ALL ON SCHEMA public FROM PUBLIC; | ||
| GRANT USAGE ON SCHEMA public TO grimoire_app; | ||
|
|
||
| -- Runtime role: no default grants. Each migration grants the runtime role exactly what it needs on the tables it creates; | ||
| -- Kysely's own bookkeeping tables (kysely_migration, kysely_migration_lock) get no grant. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.