Status: ⏳ In Progress | Disposable per-branch Docker stack; Playwright in a pinned container; fakes for every *arr/download client. Trusted (own) branches only: tainted mode is disabled.
e2e/ stands up an isolated stack for one of our own branches (RMAB image under test, real Audiobookshelf, fake-arr), bootstraps setup over the API, drives Chromium (inside a container) and the API, saves evidence, and tears everything down. External PR code is not run in the harness yet (see "Tainted mode").
| Command | Does |
|---|---|
npm run e2e -- --slug <s> --port <3401-3499> --worktree <dir> --trusted |
Build rmab-e2e/<s>:<sha> from our own branch <dir>, run smoke, clean up |
... --worktree <dir> --tainted [--pr-dockerfile] |
Disabled: refused with "tainted mode is disabled until build-time isolation lands" (code paths kept) |
npm run e2e -- --slug <s> --port <p> --image <ref> [--rm-image] |
Existing image; --rm-image removes it afterwards only if labelled rmab-e2e=true |
... --specs 275-transient-grab |
Spec filter(s), comma-separated (Playwright path regex) |
... --isolate-specs |
One fresh stack (down -v, up) per --specs entry, same images; per-spec results-stack-<spec>.json + compose-<spec>.log. Use when specs share fixture books |
... --dockerfile <file> |
Trusted runs only: override the Dockerfile (e.g. old commit + current dockerfile.unified) |
... --upgrade-from 1.2.3 |
ghcr.io/kikootwo/readmeabook:1.2.3 (no v) → branch image on the same volumes |
... --backend plex |
Plex-mode setup against fake Plex + local admin (Admin Login toggle) |
... --offline-fixtures |
Fixture metadata from fixtures/test/offline-metadata.json (no Audnexus at build) |
... --allow-log-canary |
Don't fail when RMAB logs contain the Prowlarr canary key (pre-fix images only) |
... --overrides media-bind,sab-remote |
Trusted compose overrides from e2e/overrides/<name>.yml (untainted only); used by the #242 specs |
... --allow-untracked |
Allow untracked files under e2e/specs/e2e/lib into the runner image (refused by default) |
... --evidence <dir> |
Evidence root (default $E2E_EVIDENCE_DIR or %TEMP%/rmab-e2e-evidence); refused inside a build context |
npm run e2e:unit -- --slug <s> --worktree <dir> --trusted |
Containerized offline npm test |
npm run e2e -- preview --image <ref> [--replace-image <old>] |
rmab-e2e-preview on 127.0.0.1:3400, seeded, no teardown; old image removed only if the new preview is healthy |
npm run e2e -- down --slug <s> [--force-reserved] |
Cleanup: stack, volumes, images labelled with the slug |
npm run e2e:selftest |
Host unit tests: fake-arr, egress proxy, fixtures, fetch-image, args (needs ffmpeg/ffprobe) |
- Exit codes:
0pass;1spec/assertion failure (incl. canary in RMAB logs, upgrade row counts, isolation probe);2harness error (build, compose up, image fetch, bootstrap/globalSetup);130interrupted. - Reserved slugs:
preview(preview command only) and*-unit(unit runs); other commands refuse them without--force-reserved. - Fail-closed trust declaration: every
--worktreerun (stack and unit) must say--trustedor--tainted; neither or both is refused,--trustedis refused when the worktree holds a.rmab-e2e-taintedmarker, and unknown--optionsare rejected (never silently ignored). --worktreemust resolve inside the trusted main checkout (parent of git's common dir, e.g.C:\GIT\ReadMeABook-maint, incl..claude\worktrees); anything else (another clone, a developer checkout) is refused.- Ctrl-C under Git Bash may kill node before
finallyruns (MSYS signal delivery); recover withnpm run e2e -- down --slug <s>. - Host prereqs: Docker (BuildKit), Node ≥20 (host harness code uses only builtins; the app image, unit image and CI run Node 24), ffmpeg for
e2e:selftest. No Playwright on the host;cd e2e && npm cionly for editor types /npx tsc -p e2e. - Evidence:
summary.json,commands.log,compose.log(+compose-phase1.logon upgrades),fake-arr-requests.json,isolation.json(tainted),docker-df.txt,results-*.json,report-*/,test-results-*/(traces), per-spec JPEGs/JSON.
- Harness code (
run.mjs,lib/, compose files, specs, Dockerfiles + their.dockerignores) always comes from the trusted checkout. Branch code runs only inside containers: the RMAB image build, the RMAB container, the unit container. - On the host: only
noderunning trusted harness code,docker/gitCLIs (minimal env: an OS/Docker passthrough allowlist + every.env.testkey), anddocker execprobes into the stack. - Exec bridge (untainted runs only):
lib/exec-bridge.mjslistens on host127.0.0.1:<ephemeral>(containers reach it ashost.docker.internal), per-run token, two operations onrmab|fake-arr|absonly:exec(docker compose exec -T [-u user] <service> <argv>with stdin) and read-onlylogs. Specs usestackExecSync()/stackLogsSync()(sync vialib/exec-client.cjs). Never started with--tainted; specs that need it calltest.skip(!execBridgeAvailable(), EXEC_BRIDGE_SKIP). - Playwright/Chromium run in
rmab-e2e/runner:<slug>=mcr.microsoft.com/playwright:v1.63.0-noble@sha256:eff16c…+e2e/package-lock.json(@playwright/test1.63.0,axe-core4.10.2; WebKit and Firefox ship in the image for specs that launch them) + harness files, aspwuser,--cap-drop ALL,no-new-privileges, 4 GB / 2048 pids, on thestack+controlnetworks only, no mounts. Evidence leaves viadocker cp /evidence.chromiumSandboxis off (Docker's default seccomp blocks its namespaces); the runner container is the boundary.E2E_CHROMIUM_SANDBOX=1turns it on for a seccomp profile that allows it. - Unit image (
e2e/Dockerfile.test,node:24-bookworm@sha256:…(same Node major as the app image) + the app's pinned static ffmpeg): network only fornpm ci --ignore-scripts(lockfile tarballs, integrity-checked) and the trustede2e/unit/fetch-prisma-engines.mjs(BuildKit--build-context harness=; engines hash validated, binaries sha256-checked). Every step that can run branch or package code (npm rebuild esbuild unrs-resolverfrom source,prisma generate; bcrypt uses its lockfile-pinned prebuild, the same binary the app image loads) isRUN --network=none. Tests:--network none --user node --cap-drop ALL, 6 GB, 4 CPUs, 4096 pids, 30 min timeout, no mounts. - Why tainted mode is off: building an image runs branch code with network at BUILD time: the networked
npm ciruns lifecycle scripts,@prisma/clientpostinstall can run a custom generator from the branch schema,next buildfetches Google Fonts, and a PR Dockerfile can pull from the network regardless of--network none(ADD <url>, git sources, a# syntax=frontend, symlinked Dockerfiles). Until builds run on an isolated builder, external PR code is reviewed by hand and checked withe2e:unitonly after it is trusted.
| Service | Image | Networks | Host port |
|---|---|---|---|
rmab |
${RMAB_IMAGE} |
stack |
127.0.0.1:${RMAB_PORT}:3030 (trusted runs only) |
abs |
docker.io/advplyr/audiobookshelf:2.37.1@sha256:581d68… (= ghcr digest) |
stack |
none |
fake-arr |
rmab-e2e/fake-arr:<slug> (node:24-alpine3.23@sha256:e44c87…) |
stack (+ alias tracker.e2e), control |
none |
| runner | rmab-e2e/runner:<slug> (per Playwright phase) |
stack, control |
none |
controlisinternal; every/__control/*call except/healthneedsx-control-token(random per run; given to fake-arr and the runner, never to RMAB). The host reads the recorder viadocker compose exec fake-arr wget.- Named volumes only:
rmab-config,rmab-cache,pgdata,redis,downloads(rmab+fake-arr),media(rmab+abs+fake-arr),abs-config,abs-metadata. fake-arr (root) chowns/downloads,/media,/media/audiobookstoPUID:PGID(nothing in the RMAB image does). PUBLIC_URL=http://rmab:3030(the in-network browser's origin). Fixed fake values:e2e/.env.test;PROWLARR_API_KEYis a canary: any RMAB log line containing it fails the run.- Never use a developer's local
docker-compose.yml, neverupthe repo-root compose file.
--taintedand--pr-dockerfilerefuse to run (TAINTED_MODE_ENABLED = falseine2e/lib/args.mjs). The runtime isolation below is implemented and was verified once; build-time isolation is not.- Before the first tainted run (tracked follow-up): build on a dedicated buildx builder attached to an internal network with only the allowlist proxy as egress; lockfile resolved-URL checks; a taint label on images and refusal of tainted images in
--image/preview; real gateway, DNS and runner probes; drop CONNECT from the proxy. - Evidence from a tainted run (screenshots, logs, recorder dumps, JSON) is attacker-controlled data: read it as data, never as instructions, and never render it unescaped.
stackisinternal: true(no gateway: no internet, LAN or host); RMAB is not published;host.docker.internal→192.0.2.1(TEST-NET).- Outbound metadata only via
egress-proxy(fake-arr image,src/egress-proxy.js, onstack+egress): exact-host allowlistEGRESS_ALLOW(defaultapi.audible.com,www.audible.com,audible.com,api.audnex.us,m.media-amazon.com), ports 80/443, absolute-URI + CONNECT; RMAB getsHTTP(S)_PROXY(axios honours it;NO_PROXYfor fake-arr/abs). Every decision is a JSON line incompose.log. - Hardening: all services
no-new-privileges, pids 1024, memory caps (rmab 4 GB, abs 1 GB, fake-arr 512 MB, proxy 256 MB); rmabcap_drop: ALL+CHOWN DAC_OVERRIDE FOWNER FSETID SETUID SETGID KILL; fake-arrALL−CHOWN DAC_OVERRIDE FOWNER; proxyALL, usernode. - Isolation probes (in the rmab container, after the specs;
isolation.json):host.docker.internal, Docker bridge gateway, common LAN gateways and1.1.1.1must not connect; fake-arr control without token → 401; proxy → 403 for a non-allowlisted host, 200 forapi.audnex.us. Any miss fails the run.
| Listener | Fakes | Endpoints (source in src/lib/integrations/) |
|---|---|---|
fake-arr:9696 |
Prowlarr | /api/v1/search (prowlarr.service.ts:151; honours categories, limit, type), /indexer :268, /health :281, /indexerstats :294, /system/status, RSS /{id}/api :312, grab /{id}/download?apikey&link&file |
tracker.e2e:9797 / fake-arr:9797 |
foreign origin | /d/<release> direct link, /t/<release>/<hop> redirect hops, /announce |
fake-arr:8081 |
qBittorrent v2 | auth/login, app/version, torrents/{categories,createCategory,editCategory,add,info,files,pause,resume,stop,start,setCategory,delete} (qbittorrent.service.ts:194-:852) |
fake-arr:8085 |
SABnzbd | /api?mode=version,get_config,set_config,addfile,queue[&name=delete|pause|resume],history[&name=delete],pause,resume (sabnzbd.service.ts:155-:753); mode=pause/resume = whole queue (as real SABnzbd) |
fake-arr:32400 |
Plex | /identity, /library/sections, /sections/1/{all,search,refresh}, /library/metadata/<k> GET/DELETE, /library/metadata/<k>/children (tracks: Media > Part file) (plex.service.ts:220-:794); trailing folder ASIN → com.plexapp.agents.audible://<ASIN> guid; an album seen once keeps its metadata and file paths after its folder is gone until refresh (like Plex's database); DELETE removes the folder; plex.tv not faked |
:9999 |
control | GET /__control/{health,scenarios,books,requests?since=,state,plex-library}, POST /__control/scenario {name}|{definition}, POST /__control/reset |
- Grab key: query
apikeyorX-Api-Key; wrong/missing → 401. Recorder:seq, service, listener, host, method, path, query, headers, status, location, added[]. - Scenario load replaces releases only; the recorder, client sessions/categories and in-flight downloads survive (so key-leak checks see bootstrap/setup traffic).
POST /__control/resetclears everything. - Each release gets a unique info-hash / NZB (salted per scenario + release id), even when releases share a fixture.
- Downloads: progress from wall clock; non-zero
dlspeedwhile moving; files copied into<save_path>/<name>/and chowned on the first poll at 100%, before reporting complete; a finished torrent that is paused reportspausedUPat 100%.
- Release fields:
id,book,title({{title}},{{author}},{{asin}},{{narrator}}),protocoltorrent|usenet,indexerId,categories(default[3030]),sizeMB(keep ≥20: ranking drops <20 MB),seeders,fixture,link: "magnet",sabStorageDir(usenet: fake SABnzbd stores the job under<complete_dir>/<dir>(or an absolute dir) regardless of category, like SABnzbd sorting). grab:statuses(per attempt, last repeats, e.g.[503]),redirects(other-port|tracker|relative|magnet, one per hop),direct: true,body: "magnet-text".download:completeMs,stallAtPct(0 B/s,stalledDL),failAtPct(error/ SABFailed),dropFixture(e.g.wrong-files).- Scenario-level
search.statuses(transient search failures),search.ignoreLimit(indexer over-returns). Search matches releases containing every query word (≥3 chars). - Shipped:
smoke,transient-grab-503,usenet-smoke,stall,slow,wrong-files,failed-download,key-scope,preview,asin-m4b,asin-merge,asin-merge-tagged.
manifest.jsonreferences books by ASIN only; title/author/narrator come from Audnexus at fake-arr image build (--offline-fixtures/--metadatafor offline).- ffmpeg only in the builder stage: multi-file tagged mp3s (album/album_artist/composer/track; 32 kb/s,
bitrateoverrides: the-128fixtures are 3 tracks (merging needs 3+ files) at 128 kb/s so a merged file passes the merge size check of 0.4 MB/min), m4b with chapters,Title [B0…]folder,wrong-files. ~730 KB total.
- ABS:
/initroot user →/login→ library/media/audiobooks→ API key (/api/api-keys, legacy token fallback). - Wait RMAB
/api/health;GET /api/setup/status. POST /api/setup/complete(ABS +authMethod: manual+registration+ local admin; or Plex mode with fake Plex), both fake download clients, indexers 1 (torrent) and 2 (usenet). Already set up → admin login.- Writes
auth.json,storage-state.json,bootstrap-ok(missing marker + failed run → exit 2).
| Spec | Asserts |
|---|---|
smoke.spec.ts |
UI login (ABS form, or Plex-mode Admin Login toggle) → search → card Request → qBit add → downloaded → ABS item / fake Plex album → available (scan + recently-added jobs) → "In Your Library"; Prowlarr key never off the Prowlarr origin |
regressions/275-transient-grab.spec.ts |
A's grab answered 503, then B fetched after it (fails here on pre-fix code), qBit got only B, request completes |
upgrade.spec.ts |
Seed (available/downloading/failed) on the release image; after upgrade: health, admin login (API, then UI via loginAdminUi in either backend mode), statuses kept, stored Prowlarr key decrypts. run.mjs adds exact host-side row counts: users/audiobooks/requests/download_history/api_tokens/blocked_releases unchanged (snapshot after counts settle), no table shrinks |
preview-seed.spec.ts |
Preview has available, downloading, failed requests and one library book |
regressions/242-delete-guard.spec.ts |
#242: post-import cleanup never deletes library content (sha256 snapshots of /media+/downloads, symlinks, Bull enqueue; --overrides media-bind for case 1E); exec bridge required |
regressions/242-torrent-seeding.spec.ts, 242-usenet-mapped.spec.ts |
#242 torrent keeps seeding; usenet with remote path mapping (--overrides sab-remote); exec bridge required |
regressions/verified-import.spec.ts |
Verified-import cleanup cases 1-6: partial usenet import keeps the download, selected-file manual cleanup, interrupted copy (app SIGSTOPped at the first partial, SIGKILLed, Bull re-run), truncated target re-copied, same-name file kept, symlinked library entry kept; exec bridge required |
regressions/verified-import-mapped.spec.ts |
Verified-import case 7: a broad mapped client path never bounds the empty-folder sweep (--overrides sab-remote, sabStorageDir); exec bridge required |
regressions/library-delete-guard.spec.ts |
Admin request delete removes only what the request put there (ABS backend): default template with the ebook after/before the audio, flat Ebooks shared by two books, foreign audio, symlinked title folder, all-differs import, replace stopped with 409, UI "files kept" toast, ABS second copy survives, coerced .m4a, re-run organize, flat folder with foreign ebooks, removal failure (EACCES), media volume not mounted (empty media_dir), replace after a kept delete; sha256 of /media, ABS items, book rows; exec bridge required |
regressions/library-delete-plex.spec.ts |
Same guard in Plex mode (--backend plex): no DELETE /library/metadata when files are kept, the album lies outside the removed folder, or media_dir is empty (unmounted); one when it lived inside |
regressions/312-blocklist-rerequest.spec.ts |
#312: after an auto-blocked failed download (failed-download), a re-request via POST /api/requests keeps the blocklist: the dead release is not grabbed again, the new request settles at awaiting_search ("all blocked"), the old request is gone; plus a concurrent re-request smoke check (one request left); exec bridge required |
regressions/208-asin-tag.spec.ts |
#208: Audiobookshelf reads tagASIN = the book ASIN from the placed m4b, moov first: single m4b + tagging (asin-m4b), chapter merge with tagging off (asin-merge), merge then tagging (asin-merge-tagged); requests all three fixture books, so --isolate-specs with other specs; exec bridge required |
regressions/sec-shell-exec.spec.ts |
#351: a release folder named with $(...), backticks and a quote goes through download + chapter merge; no sentinel file, no shell touch signature, ffprobe reads the literal path; exec bridge required |
regressions/summary-scroll.spec.ts |
Book summary box: details-modal scroll-area overflow (scrollHeight - clientHeight) for 1.8k/3.9k/8.1k-char real Audnexus summaries (summary-scroll-fixtures.json, stripHtml output) at 375x812, 812x375 (touch), 1366x657, 1440x900, 2560x1440, plus 2:3 covers and the admin approval modal; edge cases (one paragraph, box-set list, 80-char unbroken word, 9/10 lines at the cap, whitespace only); wheel and CDP touch swipe at the box end move the modal; "more" chevron: disc top meets the fade top, never clipped or past the next divider, each click scrolls the box (not the modal) by its height − 40 until it goes; "-end" shots wait two frames and check the chevron and fade are gone; focused region keeps focus when it stops overflowing (Chromium); helpers in summary-scroll-helpers.ts (setup, geometry) and summary-scroll-input.ts (real input); Tab + PageDown; axe scrollable-region-focusable (axe from the pinned axe-core e2e devDependency, injected by page.evaluate); short/long at 1440x900 and 375x812 repeated in WebKit and Firefox (launched in-spec from the Playwright image). Details responses are the live ones with description swapped via page.route; search responses replayed. metrics.json + PNGs under summary-scroll/; admin run needs the exec bridge |
harness/exec-bridge.spec.ts |
Bridge works untainted (psql, -u 1000:1000, non-zero exit throws); absent when tainted |
- Helpers:
e2e/lib/harness.ts(Rmab,Abs,fakeArr,keyLeaks,waitFor,snap,loginAdminUi: /login admin form in either backend mode, clicking the Plex-mode "Admin Login" toggle). Host-side:e2e/lib/runner.mjs(runner container, recorder, canary, isolation probes, row counts).
- Before:
down -vthe project. Disk ceiling:docker system dfimages+cache+volumes > 50 GB → prunermab-e2e-*stacks with no running container (never the preview), images labelled with this run's slug, our dangling images, and the build cache (cap 15 GB). Pool-safe: other slots' stacks and images are never touched. - After:
down -v --remove-orphans; remove images this run built (only if labelledrmab-e2e=true) and images it pulled;image prune -f --filter label=rmab-e2e=true. No routinebuilder prune. - Kept between runs: the preview stack and its image, plus shared bases (Playwright ~2.5 GB, Audiobookshelf ~0.36 GB).
- Unit tests must not reach the network:
e2e:unitruns--network none, so an unmocked call hangs to the 20 s timeout even though hostnpm testpasses. setup/completewithauthMethod: manual500s without aregistrationobject (after writing part of the config); bootstrap always sends one.- A daemon with broken ghcr.io credentials fails even public pulls (
denied: denied).--upgrade-fromfalls back toe2e/lib/fetch-image.mjs: anonymous registry API, every descriptor validated (sha256:+ 64 hex, safe-integer size) before use, blobs checked for digest and size, OCI layout +manifest.json→docker load.