Skip to content

Latest commit

 

History

History
627 lines (556 loc) · 33.5 KB

ARCHITECTURE.md

File metadata and controls

627 lines (556 loc) · 33.5 KB

BookFlow — Architecture

How BookFlow is put together: layered design, module responsibilities, request flows, and the invariants every change must preserve.

1. Overview

BookFlow turns existing filesystem folders into a self-hosted OPDS catalog with on-demand device optimization and multi-user reading progression.

                 READ ONLY
              ┌──────────────┐
              │ Library      │
              │ filesystem   │
              └──────┬───────┘
                     │ scan (write only to SQLite)
                     ▼
              ┌──────────────┐
              │ SQLite index │
              └──────┬───────┘
        ┌────────────┼────────────┐
        ▼            ▼            ▼
      OPDS        OPDS X3      OPDS X4
        │            │            │
     original   on-demand     on-demand
        │     epubkit/cbz2xtc epubkit/cbz2xtc
        ▼            ▼            ▼
     source       cache        cache
   EPUB / CBZ   EPUB / XTC   EPUB / XTC

Three rules define the system:

  1. The filesystem is authoritative. SQLite is only an index; books are stored as (folder.path, relative_path), never absolute paths, and the library is never written to, renamed, or deleted.
  2. Optimization is strictly on-demand. Only a request to /opdsx3/download/<id> or /opdsx4/download/<id> may invoke the epubkit or cbz2xtc pipelines — never scanning, startup, or background jobs.
  3. Multi-user authentication, single surface. Admin and reader users in SQLite (session cookie for /admin, HTTP Basic for /opds, /opdsx3 and /opdsx4), with reading progression tracked per user per book regardless of which download URL the reader used.

2. Technology

Concern Choice
Web Flask 3 (application factory, blueprints)
Templates Jinja2 (server-rendered admin UI)
Persistence SQLAlchemy 2 ORM + SQLite, Alembic migrations
Passwords argon2-cffi (Argon2id, SQLite stored password hashes)
EPUB parse stdlib zipfile + xml.etree
PDF parse pypdf
Optimization vendored epubkit (EPUB) & cbz2xtc (CBZ→XTC) (Pillow, lxml, cssutils)
XML feeds stdlib xml.etree.ElementTree
Prod server gunicorn
Quality pytest, ruff (E,F,I,UP,B,C4,SIM,RUF059, 88 cols)

No FastAPI, no Redis, no Celery, no second HTTP service.

3. Project layout

bookflow/
├── pyproject.toml / uv.lock      deps, ruff + pytest config
├── alembic.ini / migrations/     schema migrations (0001_initial_schema)
├── README.md / ARCHITECTURE.md / REFERENCE.md / AGENTS.md / DESIGN.md
├── .env.example                  documented OPDS_* variables
│
├── src/bookflow/
│   ├── app.py                    application factory + / → /admin/ + /healthz
│   ├── wsgi.py                   module-level app for gunicorn
│   ├── auto_scan.py              background rescan of every folder on a timer
│   ├── config.py                 frozen Settings dataclass (env → values)
│   ├── constants.py              shared constants (PAGE_SIZE, limits)
│   ├── utils.py                  time and timezone helper utilities
│   │
│   ├── database/
│   │   ├── database.py           engine, session_scope, PRAGMAs, reset
│   │   └── models.py             ORM tables: folders, books, users, progressions, optimized, profiles
│   │
│   ├── auth/                     admin session authentication
│   │   ├── service.py            PasswordVerifier, LoginRateLimiter, CSRF
│   │   ├── routes.py             /admin/login, /admin/logout
│   │   └── decorators.py         @login_required
│   │
│   ├── admin/                    operational UI (no file management)
│   │   └── routes.py             dashboard, folders, library browser,
│   │                             scan, download, cache clear, optimizer, health
│   │
│   ├── library/                  filesystem → index
│   │   ├── service.py            register/list/remove folders
│   │   ├── browse.py             clamped directory listing for the picker
│   │   ├── paths.py              book_file(): the single source of book paths,
│   │                             send_book_response(): stream them as downloads
│   │   ├── scanner.py            walk, reconcile, scan statistics
│   │   └── metadata.py           EPUB/PDF metadata + on-demand cover
│   │
│   ├── opds/                     catalog surface
│   │   ├── routes.py             feeds, downloads, covers, auth document
│   │   ├── generator.py          Atom/OPDS XML builders
│   │   ├── auth.py               HTTP Basic + OPDS Authentication Document
│   │   └── progression.py        OPDS Progression 1.0 GET/PUT
│   │
│   ├── optimizer/                device-specific EPUB/XTC cache
│   │   ├── service.py            optimize_book(), cache clear + prune, profile config
│   │   ├── locks.py              per-book locks (profile + progression)
│   │   ├── epubkit/              vendored pipeline (see NOTICE)
│   │   └── cbz2xtc/              vendored pipeline (see NOTICE)
│   │
│   ├── health/                   diagnostics
│   │   └── service.py            component checks + library statistics
│   │
│   ├── templates/                layout, login, dashboard, folders,
│   │                             add_folder, browse_folders, library,
│   │                             users, optimizer, health, plus the shared partials
│   │                             _checks/_breadcrumbs/_rows
│   └── static/style.css
│
└── tests/                        366 tests (see §11)

Deliberate deviations from the originally sketched layout: every ORM model lives in database/models.py (there is no library/models.py), and the epubkit integration is a package optimizer/epubkit/ rather than a single epubkit.py module.

4. Application bootstrap

src/bookflow/app.py — factory pattern, one function:

  1. Settings.from_env() (or the settings passed in by tests).
  2. _configure_logging(settings.log_level) — root logger level from OPDS_LOG_LEVEL, plus a stderr handler when the process has none.
  3. Flask config: secret key (OPDS_SESSION_SECRET, otherwise a random one generated once and persisted to data/.session_secret so restarts and gunicorn workers keep sessions valid), HttpOnly + SameSite=Lax cookies, Secure from OPDS_SESSION_COOKIE_SECURE, MAX_CONTENT_LENGTH = 1 MB.
  4. settings.ensure_directories() — create data/ and the x3/x4 cache dirs.
  5. init_engine(settings) — process-wide SQLAlchemy engine.
  6. ProxyFix(x_for=…) when OPDS_TRUSTED_PROXY_HOPS is set — rewrites request.remote_addr from X-Forwarded-For; skipped entirely (the default) when no trusted proxy is configured.
  7. Extensions (kept on app.extensions, not globals):
    • password_verifier — Argon2id hash of the configured password, built once at startup, used by both admin login and OPDS Basic Auth.
    • login_rate_limiter — in-memory failure counter per username.
  8. Jinja globals csrf_token (form templates) and is_admin (the signed-in nav in layout.html).
  9. Register four blueprints: auth, admin, opds, progression.
  10. GET / → 302 to /admin/ (convenience redirect for the browser).
  11. GET /healthz → {"status": "ok"} (public liveness probe).
  12. AutoScanner(app, settings.auto_scan_interval).start() when the interval is above zero (OPDS_AUTO_SCAN_INTERVAL, default hourly; 0 disables) — a daemon thread that rescans every folder shortly after boot and then on the period. The thread is kept at app.extensions["auto_scanner"] so tests can stop it; it never starts in the test suite, whose settings fixture sets the interval to 0.

Production servers build the app through bookflow.wsgi, whose module-level app = create_app() is what gunicorn bookflow.wsgi:app serves. Tests build apps through the build_app fixture factory (tests/conftest.py), which calls reset_engine() first and again on teardown.

5. Configuration

config.py defines a frozen Settings dataclass; every field maps to one environment variable with a default (see .env.example):

Variable Default Notes
OPDS_DATA_DIR ./data SQLite + cache root (only writable dir)
OPDS_DATABASE_URL derived from data dir override for tests/production
OPDS_SESSION_SECRET (auto-generated) unset → random secret persisted to data/.session_secret (0600), shared by all workers
OPDS_ADMIN_USERNAME / OPDS_ADMIN_PASSWORD admin / (empty) empty password → OPDS returns 503
OPDS_SESSION_COOKIE_SECURE false enable behind HTTPS
OPDS_SCAN_EXTENSIONS .epub,.pdf,.cbz,.cbr,.mobi,.azw3 scanner scope
OPDS_AUTO_SCAN_INTERVAL 3600 seconds between background rescans of every folder; 0/off disables
OPDS_BROWSE_ROOT / root the admin folder browser is clamped to (typed paths unaffected)
OPDS_TRUSTED_PROXY_HOPS 0 reverse proxies in front of the app; > 0 rewrites request.remote_addr from X-Forwarded-For (login rate limiter behind a proxy)
OPDS_LOG_LEVEL INFO root logger level applied by create_app; a stderr handler is added only when the process has none

Derived paths: cache_dir = data/cache/optimized, x3_cache_dir/x4_cache_dir beneath it. No password is ever stored — the env value is Argon2id-hashed in memory at startup.

6. Data layer

database/database.py owns a process-wide engine:

  • init_engine(settings) builds it; reset_engine() disposes it (tests).
  • session_scope() is the only way code opens a session: commit on success, rollback on exception, always close. Call sites are short and sync.
  • SQLite specifics: check_same_thread=False (Flask threads) and a connect listener that runs PRAGMA foreign_keys=ON so the schema's ON DELETE CASCADE actually applies.
  • Schema changes go through Alembic (migrations/versions/); models are verified against migrations in tests/test_database.py.

Schema (migrations 0001, 0002, 0003, 0004)

users                               books
├── id (PK)                         ├── id (PK)
├── username (UNIQUE)               ├── folder_id (FK → library_folders, CASCADE)
├── password_hash                   ├── relative_path (unique per folder)
├── is_admin                        ├── title, authors (index), publisher,
└── created_at                      │   isbn, description, series
                                    ├── file_format, file_size, file_modified_at
library_folders                     └── created_at (index), updated_at
├── id (PK)
├── path (unique, 4096)             optimized_books
├── name                            ├── id (PK)
├── created_at, updated_at          ├── book_id (FK, CASCADE)
└── last_scan_at/duration/…          ├── profile ('x3' | 'x4')
                                    ├── source_mtime, source_size   ← validity
progressions (per user + book)      ├── optimized_size
├── id (PK)                         └── UNIQUE (book_id, profile)
├── user_id (FK → users, CASCADE)
├── book_id (FK → books, CASCADE)
├── progression (float 0..1)
├── modified (datetime)
├── device_id, device_name
├── title, references (JSON)
├── created_at, updated_at
└── UNIQUE (user_id, book_id)

books.folder_id + relative_path is unique; absolute paths are never stored. optimized_books rows are the index of cache files — the cache directory itself is treated as rebuildable.

7. Module responsibilities

7.1 auth/ — admin session authentication

  • PasswordVerifier(username, password) — hashes the configured password with Argon2id at construction; verify() compares a submitted password. No plaintext is persisted or logged.
  • LoginRateLimiter — in-memory failure counts per username with a time window; repeated failures are rejected until the window expires or a successful login resets the counter.
  • CSRF: ensure_csrf_token() mints a session token (also exposed to Jinja as csrf_token()); validate_csrf() compares it to the submitted form value; require_csrf() is the before_request hook that enforces it on writes, registered by both the auth and admin blueprints. safe_next_target() restricts post-login redirects.
  • @login_required guards every admin view; unauthenticated requests redirect to /admin/login with a validated ?next=.
  • auth/routes.py: GET/POST /admin/login, POST /admin/logout.

7.2 library/ — filesystem → index

  • service.add_folder() validates a path (absolute, exists, directory, readable, normalized via resolve(), not nested inside another registered folder) and inserts a library_folders row. remove_folder() deletes only the row — files on disk are untouched. list_folders() adds the per-folder index stats (books, size, last scan) and get_folder(id) reads one row with its own stats query — both through the same _folder_data() shape.
  • browse.browse_directory(raw_path, root) lists one directory for the admin picker: read-only, clamped to OPDS_BROWSE_ROOT with resolve() + is_relative_to(), directories only, capped at 500 entries. Selection just prefills the add-folder form; registration still goes through add_folder().
  • paths.book_file(book_id) turns a book row into an on-disk path: root containment via resolve() + is_relative_to(), 404 on anything outside the root. Shared by the OPDS and admin blueprints. paths.send_book_response(target, download_name=…, mimetype=…) streams an already-resolved file as a conditional attachment (MIME type from opds.generator.book_mime_type()), so the admin original download, /opds/download/<id> and the X3/X4 optimized download all share one response builder.
  • scanner.scan_folder(folder_id, extensions):
    • a per-folder threading.Lock makes concurrent scans of the same folder fail fast with ScanInProgress → HTTP 409;
    • one short session reads the folder path and the index's view of the files; all filesystem work then runs with no session open, so a scan never holds a pooled connection across the walk;
    • recursively walks the root (symlinks skipped), filters by extension, and reads size + mtime for change detection; metadata for new or changed files is parsed before the write session opens;
    • _reconcile() (the write session) upserts new/changed books, drops deleted ones and records the last_scan_* fields;
    • missing/unreadable root: keeps the existing index and records last_scan_status='error' + last_scan_error so an unmounted disk can never wipe the catalog;
    • per-file errors are recorded without aborting the scan; produces a ScanResult (added/updated/removed/unchanged/errors/duration).
    • Never imports the optimizer.
  • metadata.extract_metadata() — dispatch on suffix:
    • EPUB: locate OPF via the container, parse with ElementTree: title, creators, publisher, ISBN (normalized), description, series; filename fallback when anything is missing or the file is corrupt.
    • PDF: embedded pypdf document info, same fallback.
    • CBZ: fallback title from filename; embedded first image used for covers.
    • extract_cover() resolves the EPUB or CBZ cover item to raw bytes — used on demand by the OPDS cover endpoint, not stored in the index.

7.3 opds/ — the catalog

  • routes.py — all catalog endpoints (table in §8). Shared helpers:
    • one blueprint serves all three catalog prefixes, so every rule carries its full path instead of a shared url_prefix; OPDS_URL_PREFIXES (/opds, /opdsx3, /opdsx4) is the list the app-level error fallback matches on;
    • before_request registers require_basic_auth() (from auth.py, shared with progression.py) with skip="opds.authentication", so HTTP Basic covers everything under those prefixes except the public authentication document;
    • book paths come from library.paths.book_file() (§7.2) — the only function that turns a book row into a path, so URL input can never escape the library;
    • blueprint-scoped @bp.errorhandler(HTTPException) renders XML errors for every status — plus unhandled exceptions, which arrive as an InternalServerError — scoped to the blueprint so admin pages keep Flask's HTML error pages. Routing failures never reach a blueprint (there is no matched URL rule, so request.blueprints is empty); create_app registers an app-level handler that answers those under OPDS_URL_PREFIXES with the same documents and leaves everything else alone;
    • every acquisition feed is built in two steps — _book_page() fetches one page (total, updated stamp, rows) and _acquisition_response() renders it — so counts, self/rel="next" links and the response type live in one place;
    • pagination: 50 entries per page with a rel="next" link.
  • generator.py — pure builders over ElementTree: navigation_feed(), acquisition_feed(), book_entry(). Each entry carries acquisition, thumbnail/image, and progression links. With a device profile, _acquisition() points EPUBs (application/epub+zip) and CBZs (application/vnd.xteink.xtc) at /opdsx3/download/<id> or /opdsx4/download/<id>, and lets other formats fall back to the original download so no link is broken.
  • auth.py — authenticate() verifies the Basic header against the shared PasswordVerifier, and require_basic_auth() is the before_request hook that applies it for a blueprint (with an optional skip endpoint); unauthorized() returns the OPDS Authentication Document (application/opds-authentication+json) as the body and a Link: rel="http://opds-spec.org/auth/document" header, plus WWW-Authenticate: Basic.
  • progression.py — OPDS Progression 1.0:
    • GET returns the stored document, or an empty valid payload when none exists;
    • PUT validates media type → JSON → required fields (modified, device, progression) → range 0..1 → timestamps (naive values are interpreted as UTC; serialization is microsecond-preserving so a replayed PUT stays equal), then applies older-than-stored → 409, equal → 200 replay, newer → store and return 201/200;
    • errors are RFC 7807 application/problem+json with registry types (…#progression-invalid-payload, …#progression-date, about:blank for 404 and any other status);
    • one progressions row per user per book — progression belongs to the logical book and user, never to a cache profile.

7.4 optimizer/ — on-demand device EPUBs and XTC comics

  • service.optimize_book(book_id, profile, source) — the on-demand generation flow:
    1. source.stat() → source_mtime (ms) + source_size;
    2. acquire profile_lock(book_id, profile) (module-level threading.Lock registry keyed by the pair — concurrent clients for the same book and profile serialize; different profiles/books run in parallel);
    3. _cache_is_valid() — cache file (<book_id>.epub or <book_id>.xtc) exists and the optimized_books row matches current mtime/size and the recorded optimized_size equals the file on disk (a missing file reports invalid);
    4. on miss: _generate() dispatches on publication format:
      • EPUB: runs process_epub(source → tmp) via epubkit
      • CBZ: runs convert_cbz_to_xtc(source → tmp) via cbz2xtc writes into cache/<profile>/.tmp/<uuid>.<ext>, verifies success + non-empty output, then atomically moves it to <book_id>.<ext>; failures delete the temp file and raise OptimizationError (→ HTTP 500 XML), never touching the source;
    5. _record() upserts the optimized_books row.
  • service.clear_optimized_cache() — admin action: empties both cache directories (EPUB and XTC files) and deletes all index rows; returns (files, rows). Scratch files of active generations are skipped; stale scratch files are swept. Sources are never touched.
  • service.clear_profile_cache(profile) — empties cache files and index rows for a single profile when its configuration is modified in /admin/optimizer.
  • service.get_device_profile_config(profile) — queries device_profiles in SQLite to dynamically supply resolution dimensions and image flags to ProcessingOptions and ConvertOptions, with fallback defaults.
  • service.prune_orphaned_cache() — drops cached EPUB and XTC files whose optimized_books row is gone, under the same profile locks.
  • epubkit/ — vendored copy of b1rdmania/epubkit (MIT, see NOTICE): epub_processor.process_epub(input, output, options, …) with ProcessingOptions(device="x3"|"x4") profiles. Excluded from ruff rules in pyproject.toml.
  • cbz2xtc/ — vendored core conversion pipeline inspired by donutboyy/cbz2xtc and tazua/cbz2xtc (MIT, see NOTICE): in-memory page extraction, spread splitting, LANCZOS aspect-ratio scaling, Floyd-Steinberg dithering, C-level 1-bit scanline packing, and native XTC binary container packaging. Excluded from ruff rules in pyproject.toml.

7.5 health/ — diagnostics

  • run_checks(settings) → five HealthCheck(component, status, detail) results: Database (real table query — catches missing migrations), Library (each registered folder exists/is a directory/is readable), Cache (dir exists + writable), epubkit (pipeline importable), OPDS (catalog routes registered). Status is ok | warn | error.
  • library_statistics() — database-only aggregates (explicitly no filesystem walks): totals, by format, per folder, cache rows/sizes per profile, progression + known devices, last successful scan, scan errors.

7.6 admin/ — operational UI

  • Blueprint-wide before_request (require_csrf(), shared with the auth blueprint) rejects any POST/PUT/PATCH/DELETE without a valid CSRF token (400) before views run.
  • Views: dashboard (_dashboard_stats() + health badges, degrades to a readable message when migrations are missing), folder CRUD + scan, the FTP-style library browser (folder cards on /admin/folders link to per-folder directory levels from the index, files link to an original-file download), POST /admin/cache/clear (flash + redirect), /admin/health. A scan and a folder removal both finish with prune_orphaned_cache(), so cached renditions of books that just left the index do not accumulate. Both stats loaders go through _stats_or_default(load, empty), which answers a broken/missing database with that page's complete empty shape plus db_error, so neither template needs its guard to render.
  • There is deliberately no upload, delete, rename, move, or metadata editing — the UI is a control surface, not a file manager.

7.7 auto_scan.py — periodic rescan

  • AutoScanner(app, interval) owns one daemon thread per process: it waits min(5s, interval) after start(), runs one cycle, then repeats every interval seconds until stop() (the Event is what the wait blocks on, so shutdown is immediate).
  • A cycle takes an exclusive, non-blocking flock on data/.auto_scan.lock — the process that does not get it skips the cycle entirely (several gunicorn workers, one scan). Platforms without fcntl simply scan.
  • It then calls the ordinary library.scanner.scan_folder() for every library_folders row: same ScanInProgress guard (a folder already being scanned manually is skipped), same read-only walk, same last_scan_* bookkeeping, then prune_orphaned_cache() inside an app context — matching what the admin Scan route does.
  • Failures are logged per folder and per cycle; a cycle that throws (a database still being migrated, a folder vanishing mid-scan) never kills the thread, the next interval retries.

8. HTTP surface

Admin & auth

Method Path Notes
GET / 302 redirect to /admin/
GET /healthz public liveness, JSON
GET/POST /admin/login CSRF on POST, rate limited
POST /admin/logout CSRF
GET /admin/ dashboard
GET /admin/folders, /admin/folders/new library root list + add form (query path prefills)
GET /admin/folders/browse server-side folder browser, clamped to OPDS_BROWSE_ROOT
POST /admin/folders register + immediate scan
POST /admin/folders/<id>/scan re-scan (409 if running)
POST /admin/folders/<id>/delete drops index rows only
POST /admin/cache/clear §28 optimization-cache reset
GET /admin/library 302 → /admin/folders (pre-merge URL)
GET /admin/library/<id> one directory level (?path= relative, from the index)
GET /admin/books/<id>/download original file, 404 outside root
GET /admin/health component checks + statistics

All /admin/* (except login) require the session cookie.

OPDS (HTTP Basic on every route except the auth document)

Path Contents
/opds root navigation: the folder view (one entry per registered folder), All Books/Recent/Authors, Search, X3/X4 Catalogs — every feed carries the search template
/opds/books, /opds/recent, /opds/search?q= acquisition feeds, 50/page
/opds/authors, /opds/authors/<name> grouped navigation + author feed
/opds/folders, /opds/folders/<id>?path= folder hierarchy: registered folders → one directory level (subfolders + books), derived from relative_path, never from the filesystem
/opds/books/<id>, /opds/download/<id>, /opds/cover/<id> single book, original file, extracted cover
/opdsx3, /opdsx3/books, /opdsx3/recent, /opdsx3/authors, /opdsx3/search?q=, /opdsx3/folders, /opdsx3/folders/<id>, /opdsx3/books/<id>, /opdsx3/download/<id> X3 catalog — root is the folder view, same as /opds; /opdsx4 mirrors every path with X4 acquisitions
/opds/authentication public OPDS Authentication Document
/opds/publications/<id>/progression GET/PUT, application/opds-progression+json

Unknown paths under /opds, /opdsx3 or /opdsx4 return XML 404 (catch-all rule), never HTML.

9. Representative request flows

Catalog browse

GET /opds/books  →  before_request: Basic auth (401 → auth document)
                 →  session_scope: page query (ilike-escaped search, LIMIT/OFFSET)
                 →  acquisition_feed(): ElementTree → bytes
                 →  Response(application/atom+xml;profile=opds-catalog;kind=acquisition)

Optimized download (the only path into epubkit)

GET /opdsx4/download/12
  → auth → book_file(12)             (DB lookup + root-resolve guard)
  → suffix must be .epub             (else 404)
  → optimize_book(12, "x4", src)
       lock(12, x4)
       cache valid?  ── yes ──► send_file(cache/x4/12.epub)
       no → process_epub → .tmp/<uuid>.epub → os.replace → upsert row
       unlock
  → send_file(..., as_attachment)

Progression update

PUT /opds/publications/12/progression  (application/opds-progression+json)
  → auth → parse/validate (type, fields, range, timestamps)
  → compare with stored.modified:  older → 409 problem+json
                                   equal → 200 (idempotent replay)
                                   newer → upsert → 201 (create) / 200 (update)

Admin scan

POST /admin/folders/<id>/scan (session + CSRF)
  → folder lock: busy → flash + 409
  → walk + metadata with no session open → one session_scope for
    reconcile + the last_scan_* fields → redirect + flash

10. Concurrency, errors, security

Concurrency. One sync worker model (Flask dev server / gunicorn sync workers); SQLAlchemy sessions are per-session_scope and short-lived. Two in-process threading.Lock registries protect shared work: per-folder scan locks (library/scanner.py) and the per-book locks in optimizer/locks.py — per-(book_id, profile) for optimization (a waiting optimizer re-checks the cache after acquiring it) and per book for the progression PUT's read→compare→write. SQLite is configured for multiple threads (check_same_thread=False + WAL-free default journal). Each process additionally runs one background rescan thread (auto_scan.py); a non-blocking flock on data/.auto_scan.lock lets exactly one worker perform a cycle, so several gunicorn workers never walk the library at the same time.

Error handling by surface.

Surface Style
/opds, /opdsx3, /opdsx4 blueprint XML handler for every HTTPException, plus the app-level fallback in create_app for routing failures that never reach a blueprint — the fallback authenticates first, so 404/405 behave like a routed request
progression RFC 7807 application/problem+json (registry type for 400/404/409, about:blank for everything else)
/admin/* Flask's default HTML error pages; forms show inline/flash messages
auth failures 401 + auth document (OPDS) or rendered login (admin)

Security invariants (enforced in code, tested in test_optimizer.py / test_opds.py / test_auth.py):

  • one admin credential, Argon2id-hashed in memory, never logged or stored;
  • session cookie: HttpOnly, SameSite=Lax, Secure opt-in; CSRF on every state-changing admin request; login rate limiting — five failures per client address + username per ten minutes, in memory and per process, keyed on request.remote_addr. Behind a reverse proxy set OPDS_TRUSTED_PROXY_HOPS (opt-in, default off) so that address comes from the X-Forwarded-For entry the trusted proxy added;
  • downloads only for books that exist in the index, and only inside their folder root (resolve() + is_relative_to);
  • output paths are derived from book_id + profile — never from raw URL input; temp files are random UUIDs inside the cache dir;
  • the library tree is read-only: scans and optimization only ever write to SQLite and data/cache/;
  • 1 MB request body cap.

11. Testing

tests/ (366 tests, uv run pytest):

  • conftest.py — temp Settings (fresh data dir + SQLite per test), the build_app factory (runs alembic upgrade head unless migrate=False, and resets the engine around the test) and the shared app/client/ root fixtures; a suite that needs another password, browse_root or an unmigrated app layers its own *_settings/app on top.
  • factories.py — make_epub() (valid container/OPF, optional cover), make_pdf(), csrf_token(client), login_admin(client, password=...), insert_books() (ORM rows, no files), alembic_config(), and the feed helpers (parse_feed, feed_entries, feed_titles, feed_links, entry_links, links_by_rel).
  • Suite ↔ subsystem: test_auth, test_library, test_scanner, test_auto_scan, test_opds, test_progression, test_optimizer, test_optimizer_admin, test_health, test_device_catalogs, plus test_app/test_config/test_database/test_users.
  • Recurring patterns: the shared feed helpers for parsing feeds and grouping links by rel; suite-local session helpers (Basic _headers, _get/_put, _scan); insert_books() for rows that don't need real files; monkeypatched counting wrappers to prove cache hits skip epubkit.

12. Development & deployment

uv sync                                # install (incl. gunicorn)
uv run flask --app bookflow.app run --port 8000   # dev server
uv run alembic upgrade head            # migrations
uv run ruff check .                    # lint
uv run pytest -q                       # tests
uv run gunicorn --bind 0.0.0.0:8000 bookflow.wsgi:app   # production

Mount library directories read-only; only data/ (SQLite + cache) needs write access. The cache directory is disposable: clearing it (UI button or rm -rf) costs only regeneration time on the next device download.

For containers, Dockerfile + compose.yml run the same stack, either from the published ghcr.io/syfq91/bookflow:latest image or a local two-stage build (uv sync → slim runtime, non-root user): a read-only library bind mount at /library (./library by default), a named volume for /app/data, automatic alembic upgrade head at start, and a /healthz healthcheck. Settings are literals in compose.yml — OPDS_ADMIN_PASSWORD is the one to set (it ships empty on purpose: no default credential), everything else is commented out and falls back to the app default; .env is not read for interpolation.