feat(drivers/mongodb): add MongoDB driver, docs, packaging, and tests

.github/workflows/ci.yml

@@ -21,6 +21,7 @@ jobs:
       - run: bash tests/staging.bash
       - run: bash tests/archive-formats.bash
       - run: bash tests/postgres-driver.bash
+      - run: bash tests/mongodb-driver.bash
       - run: bash tests/rclone-exporter.bash
       - run: bash tests/azcopy-exporter.bash
       - run: bash tests/debian-packages.bash

.github/workflows/deb.yml

@@ -30,7 +30,7 @@ jobs:
         run: |
           set -euo pipefail
           mapfile -t packages < <(find dist -maxdepth 1 -type f -name '*.deb' -print | sort)
-          test "${#packages[@]}" -eq 5
+          test "${#packages[@]}" -eq 6
           for package in "${packages[@]}"; do
             dpkg-deb --info "$package"
             dpkg-deb --contents "$package"
@@ -42,6 +42,7 @@ jobs:
           test "$("$package_root/usr/bin/backmaster" --version)" = \
             "backmaster ${{ steps.nbgv.outputs.semver2 }}"
           test -x "$package_root/usr/lib/backmaster/drivers/postgres/driver"
+          test -x "$package_root/usr/lib/backmaster/drivers/mongodb/driver"
           test -x "$package_root/usr/lib/backmaster/exporters/rclone/exporter"
           test -x "$package_root/usr/lib/backmaster/exporters/azcopy/exporter"
 

README.md

@@ -4,9 +4,9 @@ Backmaster is a modular, fleet-oriented backup orchestrator. A **driver** create
 a consistent local backup, an **exporter** publishes it, and the core coordinates
 naming, staging, distributed locking, fallback, retention, and health checks.
 
-PostgreSQL is the first bundled driver. Exporters are available for Microsoft's
-AzCopy and for rclone; adding MongoDB, mail, or a new storage service does not
-require changing the core.
+Bundled drivers cover PostgreSQL and MongoDB. Exporters are available for
+Microsoft's AzCopy and for rclone; adding another source or storage service
+does not require changing the core.
 
 ## Start here
 
@@ -17,10 +17,12 @@ require changing the core.
 | Protect and rotate credentials | [Secrets and credentials](docs/guides/secrets.md) |
 | Set up systemd services and schedules | [systemd units and timers](docs/guides/systemd.md) |
 | Back up PostgreSQL/Patroni | [PostgreSQL driver](docs/drivers/postgresql.md) |
+| Back up MongoDB | [MongoDB driver](docs/drivers/mongodb.md) |
 | Configure Azure Blob or another destination | [rclone exporter](docs/exporters/rclone.md) |
 | Use Microsoft's Azure-native transfer tool | [AzCopy exporter](docs/exporters/azcopy.md) |
 | Schedule, monitor, and maintain backups | [Operations guide](docs/guides/operations.md) |
 | Restore PostgreSQL or perform PITR | [PostgreSQL restore runbook](docs/guides/postgres-restore.md) |
+| Restore MongoDB | [MongoDB restore runbook](docs/guides/mongodb-restore.md) |
 | Diagnose a failure | [Troubleshooting](docs/guides/troubleshooting.md) |
 | Write a driver | [Driver contract](docs/drivers/index.md) |
 | Write an exporter | [Exporter contract](docs/exporters/index.md) |
@@ -34,14 +36,15 @@ sudo apt update
 sudo apt install backmaster
 ```
 
-The `backmaster` metapackage installs:
+Available packages include:
 
 | Package | Purpose |
 | --- | --- |
 | `backmaster-core` | CLI, lifecycle, staging, and systemd units |
 | `backmaster-driver-postgres` | Physical/WAL and logical PostgreSQL backups |
+| `backmaster-driver-mongodb` | Filtered, database-granular MongoDB dumps |
 | `backmaster-exporter-rclone` | Azure Blob and other rclone destinations |
-| `backmaster` | Convenience metapackage for all three components |
+| `backmaster` | Convenience metapackage for core, PostgreSQL, and rclone |
 
 Azure-only deployments may install `backmaster-exporter-azcopy` instead of the
 rclone package. The `backmaster` metapackage retains rclone as its default
@@ -148,4 +151,6 @@ staging safety, or retention.
   both base-backup and WAL cleanup can be set to `unlimited`.
 - PostgreSQL physical mode supports WAL/PITR; logical mode supports per-database
   selection.
+- MongoDB supports per-database filtering and archive or collection-file dumps;
+  it does not claim deployment-wide PITR.
 - Backups are not proven until restore drills are automated and monitored.

config/drivers/mongodb.env.example

@@ -0,0 +1,22 @@
+# Connection URI must not contain credentials or select a database.
+MONGODB_URI=mongodb://127.0.0.1:27017/
+MONGODB_AUTH_DATABASE=admin
+
+MONGODB_DUMP_FORMAT=archive
+MONGODB_FILE_NAMING=plain
+MONGODB_GZIP=true
+MONGODB_NUM_PARALLEL_COLLECTIONS=4
+MONGODB_INCLUDE_SYSTEM_DATABASES=false
+MONGODB_DUMP_DB_USERS_AND_ROLES=false
+
+# Exact, newline-delimited names. Empty include means every visible database.
+# MONGODB_DATABASE_INCLUDE=$'application\nanalytics'
+# MONGODB_DATABASE_EXCLUDE=$'scratch\ntest'
+
+# Optional mongodump read preference document or mode.
+# MONGODB_READ_PREFERENCE=secondaryPreferred
+
+# Optional TLS policy. mongodb+srv:// enables TLS by default.
+# MONGODB_TLS_CA_FILE=/etc/ssl/certs/mongodb-ca.pem
+# MONGODB_TLS_CERTIFICATE_KEY_FILE=/etc/backmaster/secrets/mongodb-client.pem
+# MONGODB_TLS_INSECURE=false

config/drivers/mongodb.secrets.env.example

@@ -0,0 +1,5 @@
+# Keep this file readable only by root and the Backmaster service identity.
+MONGODB_USERNAME=backmaster
+MONGODB_PASSWORD=REPLACE_ME
+# MONGODB_AUTH_MECHANISM=SCRAM-SHA-256
+# MONGODB_TLS_CERTIFICATE_KEY_FILE_PASSWORD=REPLACE_ME

config/instances/nsys-mongodb.env.example

@@ -0,0 +1,17 @@
+INSTANCE_NAME=production-mongodb
+DRIVER=mongodb
+EXPORTER=rclone
+NODE_NAME=axon
+
+DRIVER_CONFIG=/etc/backmaster/drivers/mongodb/production-mongodb.env
+DRIVER_SECRET_FILE=/etc/backmaster/secrets/production-mongodb-driver.env
+EXPORTER_CONFIG=/etc/backmaster/exporters/rclone/production-mongodb.env
+EXPORTER_SECRET_FILE=/etc/backmaster/secrets/production-mongodb-exporter.env
+
+STAGING_ROOT=/var/lib/backmaster
+BACKUP_ARCHIVE_FORMAT=files
+MAX_AGE_SECONDS=82800
+CONSUL_LOCK_KEY=service/backmaster/production-mongodb
+CONSUL_LOCK_TIMEOUT=30s
+BACKUP_NAME_MODE=daily-time
+BACKUP_NAME_SUFFIX_MODE=hostname

docs/drivers/index.md

@@ -8,6 +8,8 @@ storage or credentials.
 
 - [PostgreSQL](postgresql.md) supports physical base backups with WAL and
   filtered logical dumps.
+- [MongoDB](mongodb.md) supports filtered, database-granular `mongodump`
+  archives or collection-file directories.
 
 Each bundled driver page is the canonical reference for that component's
 settings, defaults, accepted values, credentials, output, dependencies, and

docs/drivers/mongodb.md

@@ -0,0 +1,209 @@
+# MongoDB driver
+
+The MongoDB driver creates logical, database-granular backups with the official
+MongoDB Database Tools. It discovers every database visible to the backup role,
+applies exact include and exclude filters, and invokes `mongodump` once per
+selected database.
+
+Two payload layouts are available:
+
+<!-- markdownlint-disable MD013 -->
+| Format | Per-database output | Best for |
+| --- | --- | --- |
+| `archive` | One `.archive` or `.archive.gz` file | Selective downloads and simple restores |
+| `directory` | A directory of BSON and metadata files | Collection-level inspection and selective restore |
+<!-- markdownlint-enable MD013 -->
+
+Archive with gzip and plain naming is the default. It produces one recognizable,
+independently downloadable file per database. This driver does not create
+storage-engine snapshots or continuous point-in-time recovery.
+
+## Configuration reference
+
+The instance selects the policy file with `DRIVER_CONFIG` and may load a
+protected `DRIVER_SECRET_FILE`. The secret file is loaded second and wins when
+both files set the same variable.
+
+<!-- markdownlint-disable MD013 -->
+| Setting | Required/default | Meaning |
+| --- | --- | --- |
+| `DRIVER_CONFIG` | required | Readable MongoDB driver policy file |
+| `DRIVER_SECRET_FILE` | empty | Optional protected credentials file |
+| `MONGODB_URI` | required | `mongodb://` or `mongodb+srv://` deployment URI; do not select a database |
+| `MONGODB_USERNAME` | empty | Authentication username; keep in the secret file |
+| `MONGODB_PASSWORD` | empty | Authentication password; keep in the secret file |
+| `MONGODB_AUTH_DATABASE` | `admin` | Authentication source passed to both tools |
+| `MONGODB_AUTH_MECHANISM` | tool default | Optional SCRAM, X.509, AWS, GSSAPI, or PLAIN mechanism supported by the installed tools |
+| `MONGODB_DUMP_FORMAT` | `archive` | `archive` for one file per database or `directory` for BSON/metadata trees |
+| `MONGODB_FILE_NAMING` | `plain` | `plain` database names or opaque `sha256` top-level names |
+| `MONGODB_GZIP` | `true` | Enable `mongodump --gzip` |
+| `MONGODB_NUM_PARALLEL_COLLECTIONS` | `4` | Positive `mongodump --numParallelCollections` value for each database |
+| `MONGODB_DATABASE_INCLUDE` | empty | Exact newline-delimited database names; empty starts with every visible database |
+| `MONGODB_DATABASE_EXCLUDE` | empty | Exact newline-delimited names removed after inclusion |
+| `MONGODB_INCLUDE_SYSTEM_DATABASES` | `false` | Include `admin`, `config`, and `local` when selected |
+| `MONGODB_DUMP_DB_USERS_AND_ROLES` | `false` | Add `--dumpDbUsersAndRoles` to each per-database dump |
+| `MONGODB_READ_PREFERENCE` | tool default | Optional mode or JSON document passed to `mongodump --readPreference` |
+| `MONGODB_TLS_CA_FILE` | empty | CA bundle; also enables TLS |
+| `MONGODB_TLS_CERTIFICATE_KEY_FILE` | empty | Client certificate/private-key PEM; also enables TLS |
+| `MONGODB_TLS_CERTIFICATE_KEY_FILE_PASSWORD` | empty | PEM password; keep in the secret file |
+| `MONGODB_TLS_INSECURE` | `false` | Disable certificate and hostname verification; diagnostic use only |
+<!-- markdownlint-enable MD013 -->
+
+Assignments are exported, so AWS IAM credentials and proxy variables supported
+by MongoDB tools may be placed in the protected secret file. Prefer workload
+identity or short-lived credentials where possible. Do not embed credentials in
+`MONGODB_URI`: command-line connection arguments may be visible to other local
+processes on systems without process isolation.
+
+`MONGODB_TLS_INSECURE=true` weakens both certificate-chain and hostname
+verification. It should not be used for production backups. A `mongodb+srv://`
+URI enables TLS by default according to MongoDB connection-string rules.
+
+## Discovery and filtering
+
+Discovery runs `listDatabases` with `authorizedDatabases: true`. The backup role
+therefore sees only databases it is permitted to list. An explicitly included
+name that is absent or unauthorized fails the backup; it is never silently
+ignored. Exclusions win over includes. System databases are removed unless
+`MONGODB_INCLUDE_SYSTEM_DATABASES=true`. A final empty selection also fails.
+
+Use ANSI-C quoting for exact multiline lists:
+
+```bash
+MONGODB_DATABASE_INCLUDE=$'application\nanalytics\nsessions'
+MONGODB_DATABASE_EXCLUDE=$'scratch\ntest'
+```
+
+Plain output names reject path separators, line breaks, `.` and `..`. SHA-256
+naming keeps database names out of top-level archive object names; the index
+always preserves the original name. Directory dumps necessarily retain the
+MongoDB-created database subdirectory inside their opaque top-level directory.
+
+## Payload contract
+
+Default archive output:
+
+```text
+payload/
+├── databases.json
+└── databases/
+    ├── analytics.archive.gz
+    └── application.archive.gz
+```
+
+Directory output:
+
+```text
+payload/
+├── databases.json
+└── databases/
+    └── application.dump/
+        └── application/
+            ├── events.bson.gz
+            └── events.metadata.json.gz
+```
+
+`databases.json` records the source topology and replica-set name, dump format,
+naming and compression policies, collection concurrency, original database
+name, and relative path of every dump. The core then adds checksums and the
+remote manifest, or wraps the complete payload in a configured whole-backup
+archive.
+
+Every `mongodump --db` invocation has its own read window. Each database dump
+can be restored independently, but a backup spanning several databases is not
+a single deployment-wide transaction or point-in-time snapshot. `--oplog`
+cannot be combined with per-database `--db` dumps and is intentionally not
+exposed. Use a MongoDB-supported deployment snapshot or dedicated backup system
+when cross-database consistency or PITR is required.
+
+## Commands
+
+<!-- markdownlint-disable MD013 -->
+| Backmaster command | Driver action |
+| --- | --- |
+| `backmaster connectivity INSTANCE` | Check tools, authenticate, and run discovery |
+| `backmaster health INSTANCE` | Do the connectivity check and report topology plus visible database count |
+| `backmaster run INSTANCE` | Discover, filter, dump, checksum, and publish through the configured exporter |
+<!-- markdownlint-enable MD013 -->
+
+The driver executable also implements the standard internal
+`prepare PAYLOAD_DIR`, `connectivitycheck`, and `healthcheck` verbs.
+
+## Dependencies and compatibility
+
+Install matching, supported releases of `mongodb-mongosh` and
+`mongodb-database-tools`. `mongodump` creates BSON data plus collection metadata
+and indexes; `mongorestore` is the corresponding restore tool. MongoDB advises
+restoring into a compatible MongoDB version or feature compatibility version.
+Queryable Encryption collections are not supported by `mongodump`.
+
+## Setup
+
+Install the driver with the core and one exporter:
+
+```bash
+sudo apt install \
+  backmaster-core \
+  backmaster-driver-mongodb \
+  backmaster-exporter-rclone
+```
+
+Copy the packaged examples:
+
+```bash
+docs=/usr/share/doc/backmaster-driver-mongodb/examples
+
+sudo install -d -m 0755 /etc/backmaster/drivers/mongodb
+sudo install -m 0644 \
+  "$docs/config/instances/nsys-mongodb.env.example" \
+  /etc/backmaster/instances.d/production-mongodb.env
+sudo install -m 0644 \
+  "$docs/config/drivers/mongodb.env.example" \
+  /etc/backmaster/drivers/mongodb/production-mongodb.env
+sudo install -m 0640 -o root -g backmaster \
+  "$docs/config/drivers/mongodb.secrets.env.example" \
+  /etc/backmaster/secrets/production-mongodb-driver.env
+```
+
+Configure the exporter under the same instance name and use a destination root
+not shared with PostgreSQL or another MongoDB instance. Grant the backup role
+`listDatabases` and read access to every selected database. Grant the additional
+user/role privileges only when `MONGODB_DUMP_DB_USERS_AND_ROLES=true`.
+
+The packaged services already run as `backmaster`, which is normally sufficient
+for TCP or SRV connections. To order the backup after a local server without
+changing its user, add matching service drop-ins:
+
+```ini
+[Unit]
+After=mongod.service
+```
+
+Apply the ordering to backup and health services if both need it. `After=` does
+not start MongoDB; add `Wants=` only if that coupling is intentional.
+
+Commission before scheduling:
+
+```bash
+sudo -u backmaster backmaster connectivity production-mongodb
+sudo systemctl start backmaster@production-mongodb.service
+sudo -u backmaster backmaster health production-mongodb
+journalctl -u backmaster@production-mongodb.service -n 200 --no-pager
+```
+
+Then inspect `databases.json`, download and restore at least one database on an
+isolated deployment, and create the timer described in the
+[systemd guide](../guides/systemd.md#create-a-backup-timer). See the
+[MongoDB restore runbook](../guides/mongodb-restore.md) for exact commands.
+
+## Sizing and load
+
+The driver dumps databases sequentially. Within each database,
+`MONGODB_NUM_PARALLEL_COLLECTIONS` controls concurrency. Increasing it can
+reduce elapsed time but raises source I/O, CPU, memory, connection, and local
+file-descriptor pressure. Measure restore speed as well as backup speed.
+
+With `BACKUP_ARCHIVE_FORMAT=files`, peak staging is approximately the completed
+payload plus metadata. Whole-backup archive modes temporarily require both the
+payload and its final archive. A failed export intentionally retains the sealed
+stage for retry.

docs/guides/configuration.md

@@ -2,7 +2,8 @@
 
 This guide covers shared instance configuration. Component-specific behavior is
 documented with the installed component: see the
-[PostgreSQL driver](../drivers/postgresql.md) and
+[PostgreSQL](../drivers/postgresql.md) or [MongoDB](../drivers/mongodb.md)
+driver and
 [rclone](../exporters/rclone.md) or [AzCopy](../exporters/azcopy.md) exporter.
 
 An instance joins one driver to one exporter. Its name is the filename below
@@ -145,6 +146,32 @@ No storage credentials belong in the driver file. The
 is the canonical list of every driver setting, PostgreSQL/libpq pass-through
 variable, default, validation rule, and mode-specific dependency.
 
+## MongoDB driver file
+
+Example: `/etc/backmaster/drivers/mongodb/production-mongodb.env`
+
+```bash
+MONGODB_URI=mongodb://127.0.0.1:27017/
+MONGODB_AUTH_DATABASE=admin
+MONGODB_DUMP_FORMAT=archive
+MONGODB_FILE_NAMING=plain
+MONGODB_GZIP=true
+MONGODB_NUM_PARALLEL_COLLECTIONS=4
+MONGODB_INCLUDE_SYSTEM_DATABASES=false
+MONGODB_DUMP_DB_USERS_AND_ROLES=false
+# MONGODB_DATABASE_INCLUDE=$'application\nanalytics'
+# MONGODB_DATABASE_EXCLUDE=$'scratch\ntest'
+```
+
+Put `MONGODB_USERNAME`, `MONGODB_PASSWORD`, private-key passwords, and cloud
+identity secrets in `DRIVER_SECRET_FILE`, not the policy file. Empty inclusion
+selects every visible non-system database. Explicit missing or unauthorized
+names fail the backup, exclusions win, and an empty final selection fails.
+
+The [MongoDB driver reference](../drivers/mongodb.md#configuration-reference)
+is the canonical setting list and documents TLS, authentication, output
+layouts, consistency, permissions, setup, sizing, and health behavior.
+
 ## Rclone exporter file
 
 Example: `/etc/backmaster/exporters/rclone/production-postgres.env`
@@ -243,3 +270,5 @@ Physical payloads contain `pg_basebackup` tar archives. Logical payloads contain
 `globals.sql.gz`, `databases.json`, and configurable custom (`.dump`) or plain
 SQL (`.sql`) dumps under `databases/`. Filenames can be SHA-256 hashes or plain
 database names; see the [PostgreSQL driver reference](../drivers/postgresql.md).
+MongoDB payloads contain `databases.json` plus one archive or BSON directory per
+selected database; see the [MongoDB driver reference](../drivers/mongodb.md).

docs/guides/index.md

@@ -8,6 +8,7 @@ Task-oriented Backmaster documentation:
 - [systemd units and timers](systemd.md)
 - [Operations](operations.md)
 - [PostgreSQL restore and PITR](postgres-restore.md)
+- [MongoDB restore](mongodb-restore.md)
 - [Troubleshooting](troubleshooting.md)
 
 Component behavior and extension contracts are documented separately under