From 14235318d973592d2b7d196b4a4b72507c389f0a Mon Sep 17 00:00:00 2001 From: kikootwo Date: Tue, 27 Oct 2026 08:54:11 -0400 Subject: [PATCH 1/3] fix: an ebook request delete removes only the ebook file that request placed The ebook organize now records the file it copied into the library (Request.placedEbookFiles: path, size, sha256), and only for a 'copied' placement. A request delete removes only recorded files that are unchanged, not recorded by another active request, and in a folder the library delete guard allows. A request with no record removes nothing; a delivered one says why. Adds one nullable column. Closes #497. Co-Authored-By: Claude Opus 5.5 --- .../admin-features/request-deletion.md | 25 +- documentation/backend/database.md | 1 + documentation/integrations/ebook-sidecar.md | 2 +- documentation/phase3/file-organization.md | 2 + documentation/testing/e2e-harness.md | 4 +- .../regressions/497-ebook-delete.spec.ts | 125 +++++++++ .../regressions/497-ebook-legacy.spec.ts | 137 +++++++++ e2e/specs/regressions/497-helpers.ts | 44 +++ .../regressions/library-delete-guard.spec.ts | 13 +- .../regressions/library-delete-helpers.ts | 5 + prisma/schema.prisma | 4 + .../processors/organize-files.processor.ts | 4 + src/lib/services/request-delete-files.ts | 78 ++++-- src/lib/services/request-delete.service.ts | 8 +- src/lib/utils/ebook-placement.ts | 167 +++++++++++ src/lib/utils/library-delete-guard.ts | 26 +- .../organize-ebook-placement.test.ts | 181 ++++++++++++ .../services/request-delete-ebook-fixture.ts | 96 +++++++ ...quest-delete.ebook-placement-guard.test.ts | 260 ++++++++++++++++++ .../request-delete.ebook-placement.test.ts | 220 +++++++++++++++ .../request-delete.library-guard.test.ts | 50 ---- tests/services/request-delete.service.test.ts | 2 +- tests/utils/ebook-placement.test.ts | 173 ++++++++++++ tests/utils/library-delete-guard.test.ts | 41 +-- 24 files changed, 1516 insertions(+), 152 deletions(-) create mode 100644 e2e/specs/regressions/497-ebook-delete.spec.ts create mode 100644 e2e/specs/regressions/497-ebook-legacy.spec.ts create mode 100644 e2e/specs/regressions/497-helpers.ts create mode 100644 src/lib/utils/ebook-placement.ts create mode 100644 tests/processors/organize-ebook-placement.test.ts create mode 100644 tests/services/request-delete-ebook-fixture.ts create mode 100644 tests/services/request-delete.ebook-placement-guard.test.ts create mode 100644 tests/services/request-delete.ebook-placement.test.ts create mode 100644 tests/utils/ebook-placement.test.ts diff --git a/documentation/admin-features/request-deletion.md b/documentation/admin-features/request-deletion.md index 9a36f82c5..dbf1e6396 100644 --- a/documentation/admin-features/request-deletion.md +++ b/documentation/admin-features/request-deletion.md @@ -48,6 +48,8 @@ model Request { **Unique Constraint:** Removed from schema, enforced in application code +**Ebook placement record (#497):** `placedEbookFiles Json? @map("placed_ebook_files")` on Request: the ebook files the request's imports copied (D2 below). Nullable, no default (added by `prisma db push`; existing rows NULL) + ### Deletion Logic Flow **Service:** `src/lib/services/request-delete.service.ts`; download handling in `src/lib/services/request-delete-download.ts` (`cleanUpRequestDownload()`, never throws) @@ -122,7 +124,19 @@ model Request { - Stale record: the stored folder is gone AND outside the current `media_dir` (library moved): nothing removed or protected; the delete clears only `filePath`/`filesHash`/`fileFormat` (no backend call, records and status kept; message says so), a replace goes on - `rm` failure: kept (`the folder was partly removed (permission denied); remove the rest manually`); `fs.rm` stops at the first error, so earlier files are gone; backend and row untouched - After removal: empty parents swept up to the nearest root (`sweepEmptyParents`) - - **Ebook requests (D2):** rules a-c on `filePath`, then unlink every ebook file directly in it if the folder passes the evidence rule, or the only one if it holds exactly one; otherwise kept (`it holds N ebook files and is not provably this book's folder (...)`). A shared `{author}` or flat `Ebooks` folder fails rule c when another book points at it. Never a sweep of other books' ebooks + - **Ebook requests (D2, #497):** only the files the request's imports recorded in `Request.placedEbookFiles` (`[{ path, size, sha256, placedAt }]`, `src/lib/utils/ebook-placement.ts`); status and the rest of the folder never count. `Audiobook.filePath` is not read. Per entry (all kept reasons joined `; `): + | Check | Kept reason | + |-------|-------------| + | entry well-formed: absolute path, no NUL, ebook extension (`isEbookFile`), safe-integer size ≥ 0, 64-hex sha256 (non-array value = one bad entry) | `the placement record could not be read` | + | path not recorded by another active (`deletedAt null`) ebook request, any book | ` is also the ebook of another request` | + | rules a-c on the file's folder (`assertSafeLibraryDelete(dir, book, { evidence: false })`; symlinked folder: target + link location) | rule a-c reasons above | + | `lstat` regular file (not a symlink/folder) | ` is no longer the file RMAB placed` | + | size and sha256 match the entry | ` changed since RMAB placed it` | + | `unlink` | ` could not be removed ()` | + - File or its folder already gone (library mounted): nothing, no reason + - No record: SQL NULL or JSON null (imported before the field, or never imported). Kept reason only when the request ever delivered (`status = 'downloaded'` or `completedAt`): `this request was imported before RMAB kept a record of the ebook files it places, so no ebook file was removed`; otherwise none (`Request deleted successfully`) + - `[]` (the import found the file already in the library: `identical`/`present`/`differs`): `the ebook was already in the library when this request imported it, so it was not removed` + - Never a folder sweep: a flat `Ebooks`/shelf folder loses only the recorded file (foreign ebooks stay); a folder another book stores still fails rule c - Not handled (needs a separate ebook location, PR2): an ebook in a separate folder (custom `ebook_path_template`) is not removed by the audiobook delete 4. **Files kept: library left exactly as it is** @@ -305,6 +319,12 @@ where: { - **Issue:** `select-torrent`, `interactive-search`, `manual-search` and `fetch-ebook` (`/api/requests/[id]/`) and admin approve/deny (`/api/admin/requests/[id]/approve`) loaded the request by id only, so a deleted request could still start a download or a search - **Fix:** each loads with `deletedAt: null` (as `select-ebook`/`interactive-search-ebook` do since #463); a deleted request answers that route's usual "Request not found" 404 +**8. Ebook Request Delete Removed the User's Own Ebook (#497)** +- **Issue:** every ebook request delete (any status, e.g. a `failed` one that never downloaded) unlinked the only ebook in `Audiobook.filePath`, or every ebook there when the folder passed the audio evidence rule: the user's own epub beside an imported audiobook went +- **Cause:** nothing recorded which file an ebook import placed; `planEbookDelete` guessed from the folder +- **Fix:** the ebook organize records the file it copied (`Request.placedEbookFiles`, only for placeFile `copied`); the delete removes only recorded, unchanged files behind rules a-c (D2 above). `planEbookDelete` removed +- **Limit:** ebook requests imported before this version have no record: their delete removes no ebook file (reason shown; remove by hand). A crash between the copy and the record write leaves that file unrecorded (kept) + ## File Structure ``` @@ -314,7 +334,8 @@ Backend: - src/lib/services/request-delete-download.ts (download in the client: remove / keep seeding / leave alone) - src/lib/utils/torrent-ownership.ts (whether RMAB added the torrent) - src/lib/services/request-delete-files.ts (library files, backend items, plex_library records) -- src/lib/utils/library-delete-guard.ts (assertSafeLibraryDelete, planEbookDelete) +- src/lib/utils/library-delete-guard.ts (assertSafeLibraryDelete) +- src/lib/utils/ebook-placement.ts (Request.placedEbookFiles: record write, read, per-file delete checks) - src/lib/services/replace-library-content.ts (replace with no active request) - src/app/api/admin/requests/[id]/route.ts (DELETE endpoint) - src/lib/processors/cleanup-seeded-torrents.processor.ts (orphaned cleanup) diff --git a/documentation/backend/database.md b/documentation/backend/database.md index 421a8cbbf..3675ce477 100644 --- a/documentation/backend/database.md +++ b/documentation/backend/database.md @@ -72,6 +72,7 @@ PostgreSQL database storing users, audiobooks, requests, downloads, configuratio - `search_attempts`, `download_attempts`, `import_attempts`, `max_import_retries` (default 5) - `auto_rerequest_count` (Int, default 0) - Automatic re-requests after a failure with no new block (shelf sync / watched lists backoff step); user requests and retries reset it to 0. See [release-blocklist.md](../admin-features/release-blocklist.md#automatic-re-requests-409) - `last_search_at`, `last_import_at`, `created_at`, `updated_at`, `completed_at` +- `placed_ebook_files` (Json, nullable) - Ebook requests: files this request's imports copied into the library, `[{ path, size, sha256, placedAt }]`; `[]` = imported but the file was already there; NULL = no record (pre-#497 or never imported). The request delete removes only these. See [request-deletion.md](../admin-features/request-deletion.md) - Unique: `(user_id, audiobook_id)` - Indexes: `user_id`, `audiobook_id`, `status`, `created_at DESC` diff --git a/documentation/integrations/ebook-sidecar.md b/documentation/integrations/ebook-sidecar.md index 87a6029d3..e447ebf1a 100644 --- a/documentation/integrations/ebook-sidecar.md +++ b/documentation/integrations/ebook-sidecar.md @@ -160,7 +160,7 @@ Ebook torrent ranking uses unified algorithm with ebook-specific scoring: ## Delete Behavior **Ebook deletion is different from audiobook deletion:** -- Only deletes ebook files (`.epub`, `.pdf`, `.mobi`, etc.) +- Deletes only the ebook files this request's imports placed (`Request.placedEbookFiles`, written by the ebook organize for a `copied` file), each only while unchanged (size + sha256) and in a folder the library delete guard allows (#497). A user's own ebook in the book folder is never removed. No record (imported before the field, or the file was already there): nothing removed, the admin sees why. See [request-deletion.md](../admin-features/request-deletion.md) D2 - Does NOT delete the title folder (audiobook files remain) - Does NOT delete from backend library (Plex/ABS) - Does NOT clear audiobook availability linkage diff --git a/documentation/phase3/file-organization.md b/documentation/phase3/file-organization.md index 16f1db56a..875232c45 100644 --- a/documentation/phase3/file-organization.md +++ b/documentation/phase3/file-organization.md @@ -326,6 +326,8 @@ When user deletes and re-requests the same audiobook: - Permissions (`applyFileMode` → `applied` / `refused` / `missing`): after a copy is published, `chmod(file_chmod)` is best-effort. mergerfs/NFS/FUSE/SMB often refuse it (EPERM, ENOTSUP, EACCES, …) though the write succeeded → placement stays `copied` with `PlaceResult.warning` (path, octal mode, code). ENOENT/ENOTDIR mean no file at the path → `failed`, never verified. Same for downloaded/cached cover art (missing = cover failed). No `chown` is done - Warnings collect in `OrganizationResult.permissionWarnings` / `EbookOrganizationResult.permissionWarnings` (never in `errors`); `reportImportStatus` logs one summary per job (`summarizePermissionWarnings`) and the request message gets `permissions could not be applied to file(s)` +**Ebook placement record (#497):** after a successful ebook organize, `recordEbookPlacement` (`src/lib/utils/ebook-placement.ts`, one call in `processEbookOrganization`) appends `{ path, size, sha256, placedAt }` of the placed target to `Request.placedEbookFiles`, only when placeFile returned `copied` (RMAB created the file; hash read from the placed bytes, after the Kindle fix). `identical`/`present`/`differs` add nothing (field becomes at least `[]`). Append-only, same path replaced by the newest entry; never fails the import (a hash or DB error is a warning and leaves the field as it was). The ebook request delete removes only these files ([request-deletion.md](../admin-features/request-deletion.md) D2). + **Stale partials:** before placing, regular files in that one target folder matching the exact pattern are removed when they belong to this target name or are older than 24 h. **`placeFile(source, target)` statuses** (records keyed on the original source path): diff --git a/documentation/testing/e2e-harness.md b/documentation/testing/e2e-harness.md index b3c12bae2..5ce9d8eb8 100644 --- a/documentation/testing/e2e-harness.md +++ b/documentation/testing/e2e-harness.md @@ -109,7 +109,7 @@ | `regressions/242-torrent-seeding.spec.ts`, `242-usenet-mapped.spec.ts` | #242 torrent keeps seeding; usenet with remote path mapping (`--overrides sab-remote`); exec bridge required | | `regressions/verified-import.spec.ts` | Verified-import cleanup cases 1-6: partial usenet import keeps the download, selected-file manual cleanup, interrupted copy (app SIGSTOPped at the first partial, SIGKILLed, Bull re-run), truncated target re-copied, same-name file kept, symlinked library entry kept; exec bridge required | | `regressions/verified-import-mapped.spec.ts` | Verified-import case 7: a broad mapped client path never bounds the empty-folder sweep (`--overrides sab-remote`, `sabStorageDir`); exec bridge required | -| `regressions/library-delete-guard.spec.ts` | Admin request delete removes only what the request put there (ABS backend): default template with the ebook after/before the audio, flat `Ebooks` shared by two books, foreign audio, symlinked title folder, all-`differs` import, replace stopped with 409, UI "files kept" toast, ABS second copy survives, coerced `.m4a`, re-run organize, flat folder with foreign ebooks, removal failure (EACCES), media volume not mounted (empty `media_dir`), replace after a kept delete; sha256 of `/media`, ABS items, book rows; exec bridge required | +| `regressions/library-delete-guard.spec.ts` | Admin request delete removes only what the request put there (ABS backend): default template with the ebook after/before the audio, flat `Ebooks` shared by two books, foreign audio, symlinked title folder, all-`differs` import, replace stopped with 409, UI "files kept" toast, ABS second copy survives, coerced `.m4a`, re-run organize, flat folder with foreign ebooks (R5: the ebook request delete removes only its recorded `Lima.epub`, `Calibre One`/`Two` stay, #497), removal failure (EACCES), media volume not mounted (empty `media_dir`), replace after a kept delete; sha256 of `/media`, ABS items, book rows; exec bridge required | | `regressions/library-delete-plex.spec.ts` | Same guard in Plex mode (`--backend plex`): no `DELETE /library/metadata` when files are kept, the album lies outside the removed folder, or `media_dir` is empty (unmounted); one when it lived inside | | `regressions/312-blocklist-rerequest.spec.ts` | #312: after an auto-blocked failed download (`failed-download`), a re-request via `POST /api/requests` keeps the blocklist: the dead release is not grabbed again, the new request settles at `awaiting_search` ("all blocked"), the old request is gone; plus a concurrent re-request smoke check (one request left); exec bridge required | | `regressions/208-asin-tag.spec.ts` | #208: Audiobookshelf reads `tagASIN` = the book ASIN from the placed m4b, moov first: single m4b + tagging (`asin-m4b`), chapter merge with tagging off (`asin-merge`), merge then tagging (`asin-merge-tagged`); requests all three fixture books, so `--isolate-specs` with other specs; exec bridge required | @@ -119,6 +119,8 @@ | `regressions/summary-scroll.spec.ts` | Book summary box: details-modal scroll-area overflow (`scrollHeight - clientHeight`) for 1.8k/3.9k/8.1k-char real Audnexus summaries (`summary-scroll-fixtures.json`, stripHtml output) at 375x812, 812x375 (touch), 1366x657, 1440x900, 2560x1440, plus 2:3 covers and the admin approval modal; edge cases (one paragraph, box-set list, 80-char unbroken word, 9/10 lines at the cap, whitespace only); wheel and CDP touch swipe at the box end move the modal; "more" chevron: disc top meets the fade top, never clipped or past the next divider, each click scrolls the box (not the modal) by its height − 40 until it goes; "-end" shots wait two frames and check the chevron and fade are gone; focused region keeps focus when it stops overflowing (Chromium); helpers in `summary-scroll-helpers.ts` (setup, geometry) and `summary-scroll-input.ts` (real input); Tab + PageDown; axe `scrollable-region-focusable` (axe from the pinned `axe-core` e2e devDependency, injected by `page.evaluate`); short/long at 1440x900 and 375x812 repeated in WebKit and Firefox (launched in-spec from the Playwright image). Details responses are the live ones with `description` swapped via `page.route`; search responses replayed. `metrics.json` + PNGs under `summary-scroll/`; admin run needs the exec bridge | | `regressions/423-takeover.spec.ts` | #423: B grabbed normally, A and C already in qBittorrent (`qbit-seed`, 2 days before the grab); indexer seeds 1 min. A/C rows `added_by_rmab=false`, B `true`; admin delete of C keeps it (`torrentsKeptNotOwned: 1`, no `torrents/delete`); seeding cleanup removes only B (row → `removed`), A/C files keep the fixture sha256, C's soft-deleted request is hard-deleted without a client call; exec bridge required. Uses all three fixture books | | `regressions/423-upgrade-takeover.spec.ts` | #423 across an upgrade (`--upgrade-from 1.2.10 --upgrade-specs 423-upgrade`): on the release image B normal + A taken over, seeding unlimited so the old cleanup (triggered once) keeps both; after the upgrade the column exists, both rows NULL; with a 1-min rule the cleanup removes B and keeps A (recorded `false`, sha256 = fixture). Not named `*upgrade.spec.ts`: the stock run's filter `upgrade.spec.ts` is a path regex and would pick it up | +| `regressions/497-ebook-delete.spec.ts` | #497, an ebook request delete removes only the ebook file that request placed (audio by manual import, ebooks by the app's organize job): (a) the user's epub in an imported title folder + a `failed` ebook request (SQL) → nothing removed, no reason; (b) a real import of `Book.epub` beside the user's `Book.azw3` → `placed_ebook_files` holds the epub (path, sha256), delete removes only it; (c) the user's `Book.epub`, then a real import of a same-name, same-size, different-bytes epub (`present`) → record `[]`, request `downloaded`, delete keeps the user's file byte for byte with the "already in the library" reason. Evidence under `library-delete/497-*`; helpers in `497-helpers.ts`; exec bridge required | +| `regressions/497-ebook-legacy.spec.ts` | #497 across an upgrade (`--upgrade-from 1.2.11 --upgrade-specs 497-ebook-legacy`): on the release image L1 a delivered ebook import, L2 the user's epub + a `failed` ebook request, L3 a `pending` ebook request; after the upgrade `placed_ebook_files` exists and is NULL on all three; L1's delete removes nothing (reason: imported before RMAB kept records), L2's removes nothing, no reason; L3 organized into its existing row (`organizeEbookInto`: no new `requests`/`audiobooks` row) records its epub and its delete removes only that. Not named `*upgrade.spec.ts` (see above) | | `regressions/434-multi-disc-import.spec.ts` | #434 (`--offline-fixtures --overrides abs-fake-metadata`): A Manual Import UI, tagging on, merge off: a disc-only folder with mixed and chapter-title names (footer "in 2 discs"), 7 flat files (`Disc 1-01.m4a`, `Kingdoms Fall.m4a`…), `Extras/` stays out, each file tagged disc `d/2` + track `n/7`, Audiobookshelf `audioFiles` by `index` in disc→track order; A2 control, tagging off: the same names alone are not in that order; B merge on, API without `selectedFiles`: chapter durations run Intro, Disc 1, Disc 2; C Bulk scan + execute: untagged `CD1/`+`CD2/` = one entry, one success; D single-folder names + `filesHash` unchanged, traversal/`Extras/` entries 400. B-D tagging off. Values read first + `expect.soft`; helpers in `434-helpers.ts`; exec bridge required | | `regressions/278-ebook-manual-import.spec.ts` | #278 (`--offline-fixtures --overrides abs-fake-metadata`): one fake-ASIN book, its audiobook imported (generated mp3) and set `available`, plus a failed ebook request; a download folder with a generated mp3 and an epub stand-in. UI: My Requests ebook card, details modal, Manual Import, cleanup on: the browser lists only the epub ("1 ebook file"), sends `requestId` + `requestType: ebook`; the epub lands in the book folder (same bytes), the ebook request is `downloaded`, the audiobook request row, the book row path/files hash and the library mp3 are unchanged, no organize job for the audiobook request, the source epub removed and the mp3 and folder kept. Values read first + `expect.soft` (`278/result.json`); exec bridge required | | `regressions/302-watch-new-releases.spec.ts` | #302 UI and persistence (the release filter itself is unit-tested): made-up author (detail + books via `page.route`); Watch dialog preselects and focuses Entire catalog, ArrowDown + Space picks New releases only, POST sends `newReleasesOnly: true`, `new_releases_since` set; a series watched over the API reads NULL; Profile cards show "New releases since …" / "Entire catalog"; Change → Save disabled until changed, Entire catalog shows the backlog note, Save PATCHes and clears the cutoff (API + psql); snapshots at 1280 px, 375 px, dark and 375 dark; exec bridge required | diff --git a/e2e/specs/regressions/497-ebook-delete.spec.ts b/e2e/specs/regressions/497-ebook-delete.spec.ts new file mode 100644 index 000000000..ba7537147 --- /dev/null +++ b/e2e/specs/regressions/497-ebook-delete.spec.ts @@ -0,0 +1,125 @@ +/** + * Component: E2E regression #497 (an ebook request delete removes only the ebook file that request placed) + * Documentation: documentation/testing/e2e-harness.md + * + * Audiobookshelf backend (default). Audiobooks are imported for real (manual import); ebook files are + * placed by the app's own organize job (an ebook request + Bull organize_files job). Each case deletes + * the ebook request through DELETE /api/admin/requests/ and checks /media by sha256. + * a. the user's own epub in an imported title folder + a failed ebook request (as the ebook search + * leaves it, SQL): delete removes nothing, no reason (the issue's steps). + * b. a real import of Book.epub beside the user's Book.azw3: the record holds the epub (its sha256); + * delete removes the epub only. + * c. the user's Book.epub in the title folder, then a real import of a same-name, same-size, + * different-bytes epub (placeFile 'present'): record [], request downloaded; delete keeps the + * user's file byte for byte and says why. + */ + +import { test, expect } from '@playwright/test'; +import { execBridgeAvailable, EXEC_BRIDGE_SKIP, Rmab } from '../../lib/harness'; +import { + PUID, PGID, sh, q, psql, sqlStr, Case, exists, setConfig, newBook, bookRow, genAudio, genEbook, importAudio, importEbook, + deleteRequest, mediaShas, sameShas, adminUserId, has, eventsSince, +} from './library-delete-helpers'; +import { ALREADY_THERE_REASON, fillFile, placedRecord, requestRow, sha256 } from './497-helpers'; +import { markJobs } from './242-helpers'; + +test.skip(!execBridgeAvailable(), EXEC_BRIDGE_SKIP); +test.describe.configure({ mode: 'default' }); + +const SRC = '/downloads/manual/e2e-497'; + +function finish(c: Case) { + expect(c.finish()).toEqual([]); +} + +async function importedBook(c: Case, rmab: Rmab, key: string, title: string, author: string, asin: string) { + const audiobookId = newBook(title, author, asin); + genAudio(`${SRC}/${key}`, ['Book.m4b']); + const requestId = await importAudio(c, rmab, audiobookId, `${SRC}/${key}`); + const folder = bookRow(audiobookId).filePath; + c.check(`${key}: imported into the library (${folder})`, !!folder && exists(folder), 'control'); + return { audiobookId, requestId, folder }; +} + +test.beforeEach(() => { + setConfig('ebook_path_template', null); + sh(`mkdir -p ${q(SRC)}\nchown ${PUID}:${PGID} /downloads/manual ${q(SRC)} 2>/dev/null || true`); +}); + +test('497a the user\'s epub beside an imported audiobook + a failed ebook request: delete removes nothing', async () => { + const c = new Case('497-a-failed-request-user-epub'); + const rmab = await Rmab.loginAdmin(); + const b = await importedBook(c, rmab, '497a', 'Ebook Delete Alpha', 'Ebook Author Alpha', 'B0E497A001'); + genEbook(b.folder, 'Ebook Delete Alpha.epub'); + const ebookReq = psql( + `insert into requests (id, user_id, audiobook_id, type, status, progress, error_message, parent_request_id, created_at, updated_at) + values (gen_random_uuid(), ${sqlStr(adminUserId())}, ${sqlStr(b.audiobookId)}, 'ebook', 'failed', 0, 'e2e fixture: no ebook found', ${sqlStr(b.requestId)}, now(), now()) returning id` + ).split('\n')[0]; + c.check(`the failed request has no record (${JSON.stringify(placedRecord(ebookReq))})`, placedRecord(ebookReq) === 'NULL', 'control'); + const before = mediaShas(); + c.save('media-before.json', before); + + const res = await deleteRequest(c, rmab, ebookReq, 'delete-ebook'); + c.check('filesDeleted false, no filesKeptReason', res.details.filesDeleted === false && !res.details.filesKeptReason, 'safety', JSON.stringify(res.details)); + c.check(`message (${res.message})`, res.message === 'Request deleted successfully', 'control'); + c.check('/media unchanged by sha256 (the user\'s epub, the audiobook)', sameShas(before, mediaShas()), 'safety'); + c.check('request soft-deleted', requestRow(ebookReq).deleted, 'control'); + finish(c); +}); + +test('497b a real ebook import beside the user\'s azw3: the record holds the epub; delete removes only it', async () => { + const c = new Case('497-b-recorded-epub-user-azw3'); + const rmab = await Rmab.loginAdmin(); + const b = await importedBook(c, rmab, '497b', 'Ebook Delete Bravo', 'Ebook Author Bravo', 'B0E497B001'); + genEbook(b.folder, 'Book.azw3'); + genEbook(`${SRC}/497b-ebook`, 'Book.epub'); + const ebookReq = await importEbook(c, b.audiobookId, `${SRC}/497b-ebook`); + const epub = `${b.folder}/Book.epub`; + const record = placedRecord(ebookReq); + c.save('record.json', record); + c.check('the epub landed in the title folder', exists(epub), 'control'); + c.check( + 'record: one entry, the placed epub with its size and sha256', + Array.isArray(record) && record.length === 1 && record[0].path === epub && record[0].sha256 === sha256(epub) && record[0].size > 0, + 'safety', + JSON.stringify(record) + ); + const before = mediaShas([epub]); + + const res = await deleteRequest(c, rmab, ebookReq, 'delete-ebook'); + c.check('filesDeleted true, no filesKeptReason', res.details.filesDeleted === true && !res.details.filesKeptReason, 'safety', JSON.stringify(res.details)); + c.check('the recorded epub is gone', !exists(epub), 'safety'); + c.check('everything else under /media unchanged (azw3, audio, cover)', sameShas(before, mediaShas()), 'safety'); + c.check('the user\'s azw3 is still there', exists(`${b.folder}/Book.azw3`), 'safety'); + finish(c); +}); + +test('497c a real import of a same-name, same-size, different-bytes epub: record [], downloaded; delete keeps the user\'s file', async () => { + const c = new Case('497-c-present-not-recorded'); + const rmab = await Rmab.loginAdmin(); + const b = await importedBook(c, rmab, '497c', 'Ebook Delete Charlie', 'Ebook Author Charlie', 'B0E497C001'); + const userEpub = `${b.folder}/Book.epub`; + fillFile(b.folder, 'Book.epub', 'the user own edition'); + fillFile(`${SRC}/497c-ebook`, 'Book.epub', 'the downloaded edition'); + const userSha = sha256(userEpub); + c.check('same size, different bytes', sh(`stat -c %s ${q(userEpub)}`).trim() === sh(`stat -c %s ${q(`${SRC}/497c-ebook/Book.epub`)}`).trim() && userSha !== sha256(`${SRC}/497c-ebook/Book.epub`), 'control'); + const mark = markJobs(); + const ebookReq = await importEbook(c, b.audiobookId, `${SRC}/497c-ebook`); + const events = eventsSince(mark); + c.check('the organizer reported the epub as already there (present: not compared)', has(events, 'Ebook already exists at target, skipping copy: Book.epub'), 'log'); + const record = placedRecord(ebookReq); + c.save('record.json', record); + c.check(`record is [] (${JSON.stringify(record)})`, Array.isArray(record) && record.length === 0, 'safety'); + const row = requestRow(ebookReq); + c.check(`request downloaded (${row.status})`, row.status === 'downloaded', 'safety'); + c.check('the user\'s epub unchanged by the import', sha256(userEpub) === userSha, 'safety'); + const before = mediaShas(); + + const res = await deleteRequest(c, rmab, ebookReq, 'delete-ebook'); + c.check('filesDeleted false', res.details.filesDeleted === false, 'safety', JSON.stringify(res.details)); + c.check(`filesKeptReason says the ebook was already there (${res.details.filesKeptReason})`, res.details.filesKeptReason === ALREADY_THERE_REASON, 'safety'); + c.check(`message (${res.message})`, res.message === `Request deleted; files kept: ${ALREADY_THERE_REASON}`, 'control'); + c.check('the user\'s epub byte for byte', sha256(userEpub) === userSha, 'safety'); + c.check('/media unchanged by sha256', sameShas(before, mediaShas()), 'safety'); + finish(c); +}); diff --git a/e2e/specs/regressions/497-ebook-legacy.spec.ts b/e2e/specs/regressions/497-ebook-legacy.spec.ts new file mode 100644 index 000000000..f66416bc6 --- /dev/null +++ b/e2e/specs/regressions/497-ebook-legacy.spec.ts @@ -0,0 +1,137 @@ +/** + * Component: E2E regression #497 across an upgrade (ebook requests imported before the placement record) + * Documentation: documentation/testing/e2e-harness.md + * + * run.mjs --upgrade-from 1.2.11 --upgrade-specs 497-ebook-legacy runs this file on the release image + * (E2E_PHASE=upgrade-seed), then on the branch image on the same volumes (upgrade-verify). + * seed: L1 an audiobook + a delivered ebook import (the release copies the epub, keeps no record); + * L2 an audiobook + the user's own epub in its folder + a failed ebook request (SQL); + * L3 an audiobook + a pending ebook request (SQL), imported only after the upgrade. + * verify: requests.placed_ebook_files exists and is NULL on all three. Deleting L1 removes nothing + * and says it was imported before RMAB kept records; deleting L2 removes nothing, no reason. + * L3's import (into the existing row: no new requests or audiobooks row) records its epub, + * and deleting L3 removes only that file. run.mjs checks the row counts are unchanged. + */ + +import { test, expect } from '@playwright/test'; +import { execBridgeAvailable, EXEC_BRIDGE_SKIP, Rmab } from '../../lib/harness'; +import { + PUID, PGID, sh, q, psql, sqlStr, Case, exists, setConfig, newBook, bookRow, genAudio, genEbook, importAudio, importEbook, + organizeEbookInto, deleteRequest, mediaShas, sameShas, adminUserId, +} from './library-delete-helpers'; +import { LEGACY_REASON, hasPlacedColumn, placedRecord, requestRow, sha256 } from './497-helpers'; + +const phase = process.env.E2E_PHASE; +const SRC = '/downloads/manual/e2e-497-legacy'; +const BOOKS = { + l1: { title: 'Ebook Legacy One', author: 'Ebook Legacy Author', asin: 'B0E497L001' }, + l2: { title: 'Ebook Legacy Two', author: 'Ebook Legacy Author', asin: 'B0E497L002' }, + l3: { title: 'Ebook Legacy Three', author: 'Ebook Legacy Author', asin: 'B0E497L003' }, +}; + +test.skip(!execBridgeAvailable(), EXEC_BRIDGE_SKIP); +test.skip(phase !== 'upgrade-seed' && phase !== 'upgrade-verify', '497-ebook-legacy only runs via run.mjs --upgrade-from --upgrade-specs'); + +function finish(c: Case) { + expect(c.finish()).toEqual([]); +} + +async function importedBook(c: Case, rmab: Rmab, key: keyof typeof BOOKS) { + const { title, author, asin } = BOOKS[key]; + const audiobookId = newBook(title, author, asin); + genAudio(`${SRC}/${key}`, ['Book.m4b']); + const requestId = await importAudio(c, rmab, audiobookId, `${SRC}/${key}`, `${key}-audio`); + const folder = bookRow(audiobookId).filePath; + c.check(`${key}: imported into the library (${folder})`, !!folder && exists(folder), 'control'); + return { audiobookId, requestId, folder }; +} + +function insertEbookRequest(audiobookId: string, parentId: string, status: string): string { + return psql( + `insert into requests (id, user_id, audiobook_id, type, status, progress, parent_request_id, created_at, updated_at) + values (gen_random_uuid(), ${sqlStr(adminUserId())}, ${sqlStr(audiobookId)}, 'ebook', ${sqlStr(status)}, 0, ${sqlStr(parentId)}, now(), now()) returning id` + ).split('\n')[0]; +} + +/** The newest active ebook request of the seeded book (the one the seed made), and the book row id. */ +function seeded(key: keyof typeof BOOKS): { requestId: string; audiobookId: string; folder: string } { + const audiobookId = psql(`select id from audiobooks where audible_asin = ${sqlStr(BOOKS[key].asin)}`); + const requestId = psql( + `select id from requests where audiobook_id = ${sqlStr(audiobookId)} and type = 'ebook' and deleted_at is null order by created_at desc limit 1` + ); + return { requestId, audiobookId, folder: bookRow(audiobookId).filePath }; +} + +test('497 upgrade-seed: ebook requests on the release image (no placement record)', async () => { + test.skip(phase !== 'upgrade-seed'); + test.setTimeout(15 * 60_000); + const c = new Case('497-legacy-seed'); + setConfig('ebook_path_template', null); + sh(`mkdir -p ${q(SRC)}\nchown ${PUID}:${PGID} /downloads/manual ${q(SRC)} 2>/dev/null || true`); + const rmab = await Rmab.loginAdmin(); + c.check('the release image has no placed_ebook_files column', !hasPlacedColumn(), 'control'); + + const l1 = await importedBook(c, rmab, 'l1'); + genEbook(`${SRC}/l1-ebook`, 'Book.epub'); + const l1Ebook = await importEbook(c, l1.audiobookId, `${SRC}/l1-ebook`, 'l1-ebook'); + c.check(`L1: the release imported the epub (${requestRow(l1Ebook).status})`, exists(`${l1.folder}/Book.epub`) && requestRow(l1Ebook).status === 'downloaded', 'control'); + + const l2 = await importedBook(c, rmab, 'l2'); + genEbook(l2.folder, 'Book.epub'); + insertEbookRequest(l2.audiobookId, l2.requestId, 'failed'); + + const l3 = await importedBook(c, rmab, 'l3'); + insertEbookRequest(l3.audiobookId, l3.requestId, 'pending'); + + c.save('media-after-seed.json', mediaShas()); + finish(c); +}); + +test('497 upgrade-verify: legacy ebook requests delete nothing; a new import records and deletes only its file', async () => { + test.skip(phase !== 'upgrade-verify'); + test.setTimeout(15 * 60_000); + const c = new Case('497-legacy-verify'); + const rmab = await Rmab.loginAdmin(); + const l1 = seeded('l1'); + const l2 = seeded('l2'); + const l3 = seeded('l3'); + c.save('seeded.json', { l1, l2, l3 }); + c.check('all three ebook requests survived the upgrade', !!l1.requestId && !!l2.requestId && !!l3.requestId, 'safety'); + c.check('requests.placed_ebook_files exists', hasPlacedColumn(), 'safety'); + for (const [key, s] of Object.entries({ l1, l2, l3 })) { + c.check(`${key}: placed_ebook_files is NULL (${JSON.stringify(placedRecord(s.requestId))})`, placedRecord(s.requestId) === 'NULL', 'safety'); + } + const l1Epub = `${l1.folder}/Book.epub`; + const before = mediaShas(); + c.save('media-before.json', before); + + const r1 = await deleteRequest(c, rmab, l1.requestId, 'delete-l1'); + c.check('L1: filesDeleted false', r1.details.filesDeleted === false, 'safety', JSON.stringify(r1.details)); + c.check(`L1: reason says imported before records (${r1.details.filesKeptReason})`, r1.details.filesKeptReason === LEGACY_REASON, 'safety'); + c.check('L1: its epub is still there', exists(l1Epub), 'safety'); + + const r2 = await deleteRequest(c, rmab, l2.requestId, 'delete-l2'); + c.check('L2: filesDeleted false, no reason (never delivered)', r2.details.filesDeleted === false && !r2.details.filesKeptReason, 'safety', JSON.stringify(r2.details)); + c.check('/media unchanged by sha256 after both legacy deletes', sameShas(before, mediaShas()), 'safety'); + + sh(`mkdir -p ${q(SRC)}\nchown ${PUID}:${PGID} /downloads/manual ${q(SRC)} 2>/dev/null || true`); + genEbook(`${SRC}/l3-ebook`, 'Book.epub'); + await organizeEbookInto(c, l3.requestId, l3.audiobookId, `${SRC}/l3-ebook`, 'l3-ebook'); + const l3Epub = `${l3.folder}/Book.epub`; + const record = placedRecord(l3.requestId); + c.save('l3-record.json', record); + c.check(`L3: request downloaded (${requestRow(l3.requestId).status})`, requestRow(l3.requestId).status === 'downloaded', 'control'); + c.check( + 'L3: the record holds the placed epub (path, sha256)', + Array.isArray(record) && record.length === 1 && record[0].path === l3Epub && record[0].sha256 === sha256(l3Epub), + 'safety', + JSON.stringify(record) + ); + const beforeL3 = mediaShas([l3Epub]); + + const r3 = await deleteRequest(c, rmab, l3.requestId, 'delete-l3'); + c.check('L3: filesDeleted true, no reason', r3.details.filesDeleted === true && !r3.details.filesKeptReason, 'safety', JSON.stringify(r3.details)); + c.check('L3: its epub is gone', !exists(l3Epub), 'safety'); + c.check('everything else under /media unchanged', sameShas(beforeL3, mediaShas()), 'safety'); + finish(c); +}); diff --git a/e2e/specs/regressions/497-helpers.ts b/e2e/specs/regressions/497-helpers.ts new file mode 100644 index 000000000..af2a6149b --- /dev/null +++ b/e2e/specs/regressions/497-helpers.ts @@ -0,0 +1,44 @@ +/** + * Component: E2E #497 helpers (an ebook request's placement record, file bytes) + * Documentation: documentation/testing/e2e-harness.md + */ + +import { asApp, psql, q, sh, sqlStr } from './library-delete-helpers'; + +export const LEGACY_REASON = 'this request was imported before RMAB kept a record of the ebook files it places, so no ebook file was removed'; +export const ALREADY_THERE_REASON = 'the ebook was already in the library when this request imported it, so it was not removed'; + +export function hasPlacedColumn(): boolean { + return psql(`select count(*) from information_schema.columns where table_name = 'requests' and column_name = 'placed_ebook_files'`) === '1'; +} + +export interface Placement { + path: string; + size: number; + sha256: string; + placedAt: string; +} + +/** requests.placed_ebook_files: 'no column', 'NULL' (SQL NULL), or the stored JSON. */ +export function placedRecord(requestId: string): 'no column' | 'NULL' | Placement[] { + if (!hasPlacedColumn()) return 'no column'; + const raw = psql(`select coalesce(placed_ebook_files::text, 'NULL') from requests where id = ${sqlStr(requestId)}`); + return raw === 'NULL' ? 'NULL' : (JSON.parse(raw) as Placement[]); +} + +/** A file of exactly `bytes` bytes, the word repeated (same size, different bytes for another word), owned by the app user. */ +export function fillFile(dir: string, name: string, word: string, bytes = 4096) { + asApp(`mkdir -p ${q(dir)}\nyes ${q(word)} | head -c ${bytes} > ${q(`${dir}/${name}`)}`); +} + +export function sha256(file: string): string { + return sh(`sha256sum ${q(file)} | cut -d' ' -f1`).trim(); +} + +export function requestRow(requestId: string): { status: string; completed: boolean; deleted: boolean } { + return JSON.parse( + psql( + `select json_build_object('status', status, 'completed', completed_at is not null, 'deleted', deleted_at is not null)::text from requests where id = ${sqlStr(requestId)}` + ) + ); +} diff --git a/e2e/specs/regressions/library-delete-guard.spec.ts b/e2e/specs/regressions/library-delete-guard.spec.ts index 330467f78..5f7a2a18a 100644 --- a/e2e/specs/regressions/library-delete-guard.spec.ts +++ b/e2e/specs/regressions/library-delete-guard.spec.ts @@ -7,7 +7,7 @@ * deletes a request through DELETE /api/admin/requests/ (or the admin table, E-h) and checks * /media by sha256, the ABS item, the book row and the API's filesKeptReason. * Cases: E-a..E-h (plan), R1 ABS second copy, R3 coerced .m4a, R4 re-run organize, R5 flat ebook - * folder with foreign ebooks, R6 removal failure (EACCES), R7 media volume not mounted (emptied), + * folder with foreign ebooks (only the recorded RMAB ebook goes, #497), R6 removal failure (EACCES), R7 media volume not mounted (emptied), * E-g2 replace after a refused delete. Plex mode: library-delete-plex.spec.ts. */ @@ -264,7 +264,7 @@ test('R4 a re-run organize of the same source (files already in place): delete r finish(c); }); -test('R5 ebook delete in a flat folder holding one RMAB ebook and foreign ebooks: refused; every ebook stays', async () => { +test('R5 ebook delete in a flat folder holding one RMAB ebook and foreign ebooks: only the RMAB ebook goes (#497)', async () => { const c = new Case('R5-flat-folder-foreign-ebooks'); const rmab = await Rmab.loginAdmin(); const b = await importedBook(c, rmab, 'r5', 'Guard Book Lima', 'Guard Author Lima', 'B0LDGTRL01'); @@ -274,11 +274,14 @@ test('R5 ebook delete in a flat folder holding one RMAB ebook and foreign ebooks setConfig('ebook_path_template', 'Shelf'); genEbook(`${SRC}/r5-ebook`, 'Lima.epub'); const ebookReq = await importEbook(c, b.audiobookId, `${SRC}/r5-ebook`); - const before = mediaShas(); + const lima = `${shelf}/Lima.epub`; + c.check('Lima.epub placed in the shelf', exists(lima), 'control'); + const before = mediaShas([lima]); const res = await deleteRequest(c, rmab, ebookReq, 'delete-ebook'); - expectKept(c, 'delete-ebook', res, 'holds 3 ebook files'); - c.check('/media unchanged (all three ebooks, the audio folder)', sameShas(before, mediaShas()), 'safety'); + c.check('delete-ebook: filesDeleted true, no filesKeptReason', res.details.filesDeleted === true && !res.details.filesKeptReason, 'safety', JSON.stringify(res.details)); + c.check('Lima.epub (placed by this request) removed', !exists(lima), 'safety'); + c.check('everything else under /media unchanged (Calibre One and Two, the audio folder)', sameShas(before, mediaShas()), 'safety'); finish(c); }); diff --git a/e2e/specs/regressions/library-delete-helpers.ts b/e2e/specs/regressions/library-delete-helpers.ts index 2c38c4f9f..75ce6fe19 100644 --- a/e2e/specs/regressions/library-delete-helpers.ts +++ b/e2e/specs/regressions/library-delete-helpers.ts @@ -112,6 +112,11 @@ export async function importEbook(c: Case, audiobookId: string, downloadPath: st `insert into requests (id, user_id, audiobook_id, type, status, progress, created_at, updated_at) values (gen_random_uuid(), ${sqlStr(adminUserId())}, ${sqlStr(audiobookId)}, 'ebook', 'downloaded', 100, now(), now()) returning id` ).split('\n')[0]; + return organizeEbookInto(c, reqId, audiobookId, downloadPath, label); +} + +/** Queue the organize_files job for an existing ebook request (no new requests row) and wait for it. */ +export async function organizeEbookInto(c: Case, reqId: string, audiobookId: string, downloadPath: string, label = 'ebook') { const payload = { requestId: reqId, audiobookId, downloadPath, cleanupSource: false }; const bullId = `e2e-libdel-${Date.now()}-${Math.floor(Math.random() * 1e6)}`; const mark = markJobs(); diff --git a/prisma/schema.prisma b/prisma/schema.prisma index b6c9318fb..9df1cd0b4 100644 --- a/prisma/schema.prisma +++ b/prisma/schema.prisma @@ -248,6 +248,10 @@ model Request { // Ebook requests are created automatically when an audiobook is organized (if ebook downloads enabled) type String @default("audiobook") // 'audiobook' | 'ebook' parentRequestId String? @map("parent_request_id") // Links ebook request to originating audiobook request + // Ebook files this request put in the library (#497): [{ path, size, sha256, placedAt }], written only when + // the import copied the file (placeFile 'copied'); [] when the import found it already there. NULL = no record + // (imported before this field existed, or never imported). A request delete removes only these files. + placedEbookFiles Json? @map("placed_ebook_files") // Soft delete support deletedAt DateTime? @map("deleted_at") diff --git a/src/lib/processors/organize-files.processor.ts b/src/lib/processors/organize-files.processor.ts index caa0f8a1a..4061d9b8c 100644 --- a/src/lib/processors/organize-files.processor.ts +++ b/src/lib/processors/organize-files.processor.ts @@ -32,6 +32,7 @@ import { createAudibleDetailsLoader, readMetadataTaggingEnabled, resolveTagDetai import { applyPostImportCategory } from './post-import-category'; import { preferredEbookFile } from '../utils/import-inventory'; import { findEbookOnDisk } from '../utils/ebook-on-disk'; +import { recordEbookPlacement } from '../utils/ebook-placement'; /** * Process organize files job @@ -927,6 +928,9 @@ async function processEbookOrganization( logger.info(`Successfully moved ebook to ${result.targetPath}`); + // #497: record the ebook file this import placed (only a 'copied' file is RMAB's); never throws + await recordEbookPlacement(requestId, result.record, logger); + // The organizer reads a temp copy when the Kindle fix ran; the original is epubFilePath const importedFile = fixedEpubPath ? epubFilePath : result.sourceFile; const ebookEvidence = await buildEbookEvidence(result, importedFile ?? undefined, downloadPath); diff --git a/src/lib/services/request-delete-files.ts b/src/lib/services/request-delete-files.ts index 8cc5a3474..d19195900 100644 --- a/src/lib/services/request-delete-files.ts +++ b/src/lib/services/request-delete-files.ts @@ -3,8 +3,8 @@ * Documentation: documentation/admin-features/request-deletion.md * * Removes what a deleted request put in the library, behind the library - * delete guard: the audiobook's stored folder (audiobook requests) or ebook - * files in it (ebook requests). Backend items (Audiobookshelf, Plex) are + * delete guard: the audiobook's stored folder (audiobook requests) or the ebook + * files the request's imports recorded (ebook requests). Backend items (Audiobookshelf, Plex) are * deleted only after RMAB removed the folder itself, and only when the item's * files lived inside that folder; ABS is never asked to delete files. */ @@ -14,13 +14,17 @@ import * as path from 'path'; import { prisma } from '../db'; import { RMABLogger } from '../utils/logger'; import { canonicalPath, UnsafeDeleteError } from '../utils/delete-guard'; -import { assertSafeLibraryDelete, isStaleStoredFolder, planEbookDelete, type LibraryBook } from '../utils/library-delete-guard'; +import { assertSafeLibraryDelete, isStaleStoredFolder, type LibraryBook } from '../utils/library-delete-guard'; +import { otherRecordedPaths, planPlacedEbookDelete, readPlacements } from '../utils/ebook-placement'; import { sweepEmptyParents } from '../processors/organize-cleanup-guard'; import { isPathBelowBoundary, normalizePath } from '../utils/cleanup-helpers'; const logger = RMABLogger.create('RequestDelete'); const NO_FOLDER_REASON = 'RMAB has no stored folder for this book, so its library files were not removed'; +const LEGACY_EBOOK_REASON = 'this request was imported before RMAB kept a record of the ebook files it places, so no ebook file was removed'; +const EBOOK_ALREADY_THERE_REASON = 'the ebook was already in the library when this request imported it, so it was not removed'; +const UNREADABLE_RECORD_REASON = 'the placement record could not be read'; export interface DeleteBook extends LibraryBook { title: string; @@ -89,34 +93,56 @@ export async function removeAudiobookFolder(book: DeleteBook): Promise { - if (!book.filePath) { - logger.warn(`Skipping ebook file deletion for "${book.title}": no stored file path`); - return { deleted: 0 }; +/** The ebook request fields the D2 delete reads (scalars deleteRequest already loads). */ +export interface EbookDeleteRequest { + id: string; + status: string; + completedAt: Date | null; + placedEbookFiles: unknown; +} + +/** + * D2 (#497): unlink only the ebook files this request's imports placed + * (Request.placedEbookFiles), each only while it is still the exact file RMAB + * placed and its folder passes rules a-c. No record removes nothing. + */ +export async function removeEbookFiles(book: DeleteBook, request: EbookDeleteRequest): Promise<{ deleted: number; keptReason?: string }> { + const record = readPlacements(request.placedEbookFiles); + if (!record) { + const delivered = request.status === 'downloaded' || !!request.completedAt; + logger.info(`No ebook placement record for request ${request.id} ("${book.title}"); no ebook file removed`); + return delivered ? { deleted: 0, keptReason: LEGACY_EBOOK_REASON } : { deleted: 0 }; } - let files: string[]; - try { - files = await planEbookDelete(book.filePath, book); - } catch (error) { - const reason = refusal(error); - logger.warn(`Keeping ebook files in ${book.filePath} for "${book.title}": ${reason}`); - return { deleted: 0, keptReason: reason }; + if (record.entries.length === 0 && record.unreadable === 0) { + logger.info(`Request ${request.id} placed no ebook file ("${book.title}": the library already had it); none removed`); + return { deleted: 0, keptReason: EBOOK_ALREADY_THERE_REASON }; } + + const reasons: string[] = record.unreadable > 0 ? [UNREADABLE_RECORD_REASON] : []; + const otherPaths = record.entries.length > 0 ? await otherRecordedPaths(request.id) : new Set(); let deleted = 0; - for (const file of files) { - try { - await fs.unlink(file); - logger.info(`Deleted ebook file: ${path.basename(file)}`); - deleted++; - } catch (error) { - const code = (error as NodeJS.ErrnoException).code ?? 'error'; - const why = code === 'EACCES' || code === 'EPERM' ? 'permission denied' : code; - return { deleted, keptReason: `${path.basename(file)} could not be removed (${why})` }; + for (const entry of record.entries) { + const decision = await planPlacedEbookDelete(entry, book, otherPaths); + if (decision.kind === 'kept') { + reasons.push(decision.reason); + } else if (decision.kind === 'gone') { + logger.info(`Recorded ebook file already gone: ${entry.path}`); + } else { + try { + await fs.unlink(decision.file); + logger.info(`Deleted ebook file: ${decision.file}`); + deleted++; + } catch (error) { + const code = (error as NodeJS.ErrnoException).code ?? 'error'; + const why = code === 'EACCES' || code === 'EPERM' ? 'permission denied' : code; + reasons.push(`${path.basename(decision.file)} could not be removed (${why})`); + } } } - logger.info(`Deleted ${deleted} ebook file(s) from: ${book.filePath}`); - return { deleted }; + if (reasons.length === 0) return { deleted }; + const keptReason = reasons.join('; '); + logger.warn(`Keeping ebook file(s) of request ${request.id} ("${book.title}"): ${keptReason}`); + return { deleted, keptReason }; } /** True when `candidate` equals or sits inside the canonical `folder`. */ diff --git a/src/lib/services/request-delete.service.ts b/src/lib/services/request-delete.service.ts index 1900980bb..d99e03bb9 100644 --- a/src/lib/services/request-delete.service.ts +++ b/src/lib/services/request-delete.service.ts @@ -57,7 +57,9 @@ export interface DeleteRequestResult { * * Logic (ebook requests): * 1. Check if request exists and is not already deleted - * 2. Delete ebook files only, when the guard allows it (leave audiobook files intact) + * 2. Delete only the ebook files this request's imports recorded (Request.placedEbookFiles), + * each still unchanged and in a folder the guard allows (audiobook files never touched). + * No record: nothing removed (filesKeptReason says why for a delivered request) * 3. Soft delete request (set deletedAt, deletedBy) * Note: No backend library deletion or audiobook linkage clearing for ebooks */ @@ -127,13 +129,13 @@ export async function deleteRequest( // 3. Library files, behind the library delete guard // Audiobooks: the stored folder, only when it provably holds this import - // Ebooks: ebook files in it (leave audiobook files intact), never a shared-folder sweep + // Ebooks: only the ebook files this request's imports placed (#497), never a folder sweep const book = request.audiobook; let filesDeleted = false; let filesKeptReason: string | undefined; let outcome: LibraryFilesOutcome | null = null; if (isEbook) { - const ebook = await removeEbookFiles(book); + const ebook = await removeEbookFiles(book, request); filesDeleted = ebook.deleted > 0; filesKeptReason = ebook.keptReason; } else { diff --git a/src/lib/utils/ebook-placement.ts b/src/lib/utils/ebook-placement.ts new file mode 100644 index 000000000..fdd0c4005 --- /dev/null +++ b/src/lib/utils/ebook-placement.ts @@ -0,0 +1,167 @@ +/** + * Component: Ebook Placement Record (which ebook files a request put in the library, #497) + * Documentation: documentation/admin-features/request-deletion.md + * + * `Request.placedEbookFiles` lists the ebook files an ebook request's imports + * placed: `[{ path, size, sha256, placedAt }]`. An entry is written only when + * placeFile copied the file ('copied'); 'identical', 'present' and 'differs' + * mean the library already had it, so the import adds nothing (the field is then + * at least []). NULL means no record (never imported, or imported before this + * field existed). A request delete removes only recorded files, and only while + * each is still the exact file RMAB placed (size + sha256) and its folder passes + * library delete guard rules a-c. + */ + +import { createHash } from 'crypto'; +import { createReadStream } from 'fs'; +import * as fs from 'fs/promises'; +import * as path from 'path'; +import { prisma } from '../db'; +import type { RMABLogger } from './logger'; +import { UnsafeDeleteError } from './delete-guard'; +import { isEbookFile, type PlacementRecord } from './import-inventory'; +import { assertSafeLibraryDelete, type LibraryBook } from './library-delete-guard'; + +export interface EbookPlacement { + /** Absolute library path of the placed file (placeFile's target). */ + path: string; + size: number; + /** sha256 (hex) of the placed bytes. */ + sha256: string; + /** ISO time of the import that placed it. */ + placedAt: string; +} + +export interface PlacementRecordRead { + entries: EbookPlacement[]; + /** Items that are not a well-formed entry: their file is never removed. */ + unreadable: number; +} + +const SHA256 = /^[0-9a-f]{64}$/; + +function isPlacement(value: unknown): value is EbookPlacement { + if (!value || typeof value !== 'object' || Array.isArray(value)) return false; + const v = value as Record; + return ( + typeof v.path === 'string' && + path.isAbsolute(v.path) && + !v.path.includes('\0') && + isEbookFile(path.basename(v.path)) && + Number.isSafeInteger(v.size) && + (v.size as number) >= 0 && + typeof v.sha256 === 'string' && + SHA256.test(v.sha256) + ); +} + +/** The record as stored; null when there is none (SQL NULL and JSON null alike). */ +export function readPlacements(value: unknown): PlacementRecordRead | null { + if (value === null || value === undefined) return null; + if (!Array.isArray(value)) return { entries: [], unreadable: 1 }; + const entries = value.filter(isPlacement); + return { entries, unreadable: value.length - entries.length }; +} + +async function sha256File(file: string): Promise { + const hash = createHash('sha256'); + for await (const chunk of createReadStream(file)) hash.update(chunk as Buffer); + return hash.digest('hex'); +} + +/** The entry for a file RMAB just placed; throws unless it is a regular file. */ +export async function placementEntry(file: string): Promise { + const stat = await fs.lstat(file); + if (!stat.isFile()) throw new Error(`${path.basename(file)} is not a regular file`); + return { path: path.resolve(file), size: stat.size, sha256: await sha256File(file), placedAt: new Date().toISOString() }; +} + +/** Append `entry` to the stored value (same path: newest wins); anything stored is kept. */ +export function mergePlacements(stored: unknown, entry: EbookPlacement | null): unknown[] { + const existing = Array.isArray(stored) ? stored : stored === null || stored === undefined ? [] : [stored]; + if (!entry) return existing; + const samePath = (item: unknown) => !!item && typeof item === 'object' && (item as { path?: unknown }).path === entry.path; + return [...existing.filter((item) => !samePath(item)), entry]; +} + +/** + * After a successful ebook organize: record the file it placed, if it copied one. + * Never throws. On a failure the field is left as it was (logged): the file is + * then never removed by a request delete, which is the safe side. + */ +export async function recordEbookPlacement(requestId: string, record: PlacementRecord | undefined, logger: RMABLogger): Promise { + try { + const entry = record?.status === 'copied' ? await placementEntry(record.target) : null; + const row = await prisma.request.findUnique({ where: { id: requestId }, select: { placedEbookFiles: true } }); + await prisma.request.update({ + where: { id: requestId }, + data: { placedEbookFiles: mergePlacements(row?.placedEbookFiles, entry) as object[] }, + }); + logger.info( + entry + ? `Recorded the placed ebook file ${path.basename(entry.path)} for request ${requestId}` + : `No ebook file placed by this import (${record?.status ?? 'no placement'}); nothing recorded for a later delete` + ); + } catch (error) { + logger.warn( + `Could not record the placed ebook file for request ${requestId} (${error instanceof Error ? error.message : String(error)}); deleting this request will keep it` + ); + } +} + +/** + * Library paths recorded by every other active request, of any book. A file two + * requests recorded stays until neither needs it. Only the ebook organize writes + * records, so only ebook requests can hold one. + */ +export async function otherRecordedPaths(requestId: string): Promise> { + const rows = await prisma.request.findMany({ + where: { id: { not: requestId }, deletedAt: null, type: 'ebook' }, + select: { placedEbookFiles: true }, + }); + const paths = new Set(); + for (const row of rows) { + for (const entry of readPlacements(row.placedEbookFiles)?.entries ?? []) paths.add(path.resolve(entry.path)); + } + return paths; +} + +export type PlacedFileDecision = + /** Unlink this path: still the file RMAB placed, in a folder rules a-c allow. */ + | { kind: 'remove'; file: string } + /** Nothing there any more (or its folder is gone): nothing to do. */ + | { kind: 'gone' } + | { kind: 'kept'; reason: string }; + +/** Whether one recorded file may be unlinked for `book`'s request delete. */ +export async function planPlacedEbookDelete( + entry: EbookPlacement, + book: LibraryBook, + otherPaths: Set +): Promise { + const name = path.basename(entry.path); + if (otherPaths.has(path.resolve(entry.path))) return { kind: 'kept', reason: `${name} is also the ebook of another request` }; + + let safe; + try { + safe = await assertSafeLibraryDelete(path.dirname(entry.path), book, { evidence: false }); + } catch (error) { + if (error instanceof UnsafeDeleteError) return { kind: 'kept', reason: error.reason }; + return { kind: 'kept', reason: error instanceof Error ? error.message : String(error) }; + } + if (!safe.exists) return { kind: 'gone' }; + + const file = path.join(safe.canonicalPath, name); + try { + const stat = await fs.lstat(file); + if (!stat.isFile()) return { kind: 'kept', reason: `${name} is no longer the file RMAB placed` }; + if (stat.size !== entry.size || (await sha256File(file)) !== entry.sha256) { + return { kind: 'kept', reason: `${name} changed since RMAB placed it` }; + } + } catch (error) { + const code = (error as NodeJS.ErrnoException).code ?? 'error'; + if (code === 'ENOENT' || code === 'ENOTDIR') return { kind: 'gone' }; + return { kind: 'kept', reason: `${name} could not be checked (${code})` }; + } + return { kind: 'remove', file }; +} diff --git a/src/lib/utils/library-delete-guard.ts b/src/lib/utils/library-delete-guard.ts index 927f5ebdd..3a736f543 100644 --- a/src/lib/utils/library-delete-guard.ts +++ b/src/lib/utils/library-delete-guard.ts @@ -3,7 +3,8 @@ * Documentation: documentation/admin-features/request-deletion.md * * Decides whether deleting a request may remove the book's stored library - * folder (`Audiobook.filePath`), or ebook files in it. A folder is removed only + * folder (`Audiobook.filePath`). An ebook request delete runs rules a-c on the + * folder of each file it recorded (utils/ebook-placement.ts). A folder is removed only * when it provably holds this book's import and nothing else: * a. strictly inside media_dir (never media_dir itself) * b. equals or contains no configured root (download dirs, mapped paths, bookdrop) @@ -31,7 +32,7 @@ import { UnsafeDeleteError, } from './delete-guard'; import { generateFilesHash } from './files-hash'; -import { isAudioFile, isEbookFile, walkFiles } from './import-inventory'; +import { isAudioFile, walkFiles } from './import-inventory'; export interface LibraryBook { id: string; @@ -216,24 +217,3 @@ export async function assertSafeLibraryDelete( } return { deletePath, canonicalPath: target.path, exists: true, roots }; } - -/** - * Ebook files an ebook request delete may unlink from `filePath`: rules a-c on - * the folder, then every ebook file directly in it when the folder passes the - * audio evidence rule, or the only one when it holds exactly one. Otherwise - * (a shared or user folder with several ebooks) UnsafeDeleteError. - */ -export async function planEbookDelete(filePath: string, book: LibraryBook): Promise { - const safe = await assertSafeLibraryDelete(filePath, book, { evidence: false }); - if (!safe.exists) return []; - const entries = await fs.readdir(safe.canonicalPath, { withFileTypes: true }).catch((error: NodeJS.ErrnoException) => { - throw new UnsafeDeleteError(filePath, `the folder could not be read (${error.code ?? 'error'})`); - }); - const ebooks = entries.filter((e) => e.isFile() && isEbookFile(e.name)).map((e) => path.join(safe.canonicalPath, e.name)); - if (ebooks.length <= 1) return ebooks; - const blocker = await folderEvidenceBlocker(safe.canonicalPath, book.filesHash); - if (blocker) { - throw new UnsafeDeleteError(filePath, `it holds ${ebooks.length} ebook files and is not provably this book's folder (${blocker})`); - } - return ebooks; -} diff --git a/tests/processors/organize-ebook-placement.test.ts b/tests/processors/organize-ebook-placement.test.ts new file mode 100644 index 000000000..0f4bf5c58 --- /dev/null +++ b/tests/processors/organize-ebook-placement.test.ts @@ -0,0 +1,181 @@ +/** + * Component: Ebook Organize Placement Record Tests (#497, real organizer and filesystem) + * Documentation: documentation/admin-features/request-deletion.md + * + * processOrganizeFiles for an ebook request with the real FileOrganizer.organizeEbook and + * placeFile into a temp library: Request.placedEbookFiles gets an entry only when the import + * copied the file (copied), and stays [] when the file was already there (present, identical, + * differs). A record failure never fails the import. + */ + +import { createHash } from 'crypto'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'; +import { createPrismaMock } from '../helpers/prisma'; + +const prismaMock = createPrismaMock(); +const jobQueueMock = vi.hoisted(() => ({ addNotificationJob: vi.fn(() => Promise.resolve()) })); +const configMock = vi.hoisted(() => ({ getBackendMode: vi.fn(), get: vi.fn() })); +const organizerHolder = vi.hoisted(() => ({ media: '', temp: '' })); + +vi.mock('@/lib/db', () => ({ prisma: prismaMock })); +vi.mock('@/lib/utils/file-organizer', async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, getFileOrganizer: async () => new actual.FileOrganizer(organizerHolder.media, organizerHolder.temp) }; +}); +vi.mock('@/lib/services/library', () => ({ getLibraryService: () => ({ triggerLibraryScan: vi.fn() }) })); +vi.mock('@/lib/services/config.service', () => ({ getConfigService: () => configMock })); +vi.mock('@/lib/services/job-queue.service', () => ({ getJobQueueService: () => jobQueueMock })); +vi.mock('@/lib/services/download-client-manager.service', () => ({ + getDownloadClientManager: () => ({ + getAllClients: async () => [], + getClientForProtocol: async () => null, + getClientServiceForProtocol: async () => null, + }), +})); + +let root: string; +let source: string; +let target: string; +/** What the request row holds in placed_ebook_files before this import. */ +let existingRecord: unknown; + +const sha = (data: string) => createHash('sha256').update(data).digest('hex'); + +async function runOrganize(cleanupSource: boolean) { + const { processOrganizeFiles } = await import('@/lib/processors/organize-files.processor'); + return processOrganizeFiles({ requestId: 'req-eb', audiobookId: 'ab-1', jobId: 'job-1', downloadPath: source, cleanupSource }); +} + +/** The placedEbookFiles values the import wrote (one per write). */ +function recordWrites(): unknown[] { + return prismaMock.request.update.mock.calls + .map(([args]) => (args as { data: Record }).data) + .filter((data) => 'placedEbookFiles' in data) + .map((data) => data.placedEbookFiles); +} + +function expectDelivered() { + expect(prismaMock.request.update).toHaveBeenCalledWith({ where: { id: 'req-eb' }, data: expect.objectContaining({ status: 'downloaded' }) }); +} + +describe('ebook organize: placement record (real FileOrganizer + placeFile)', () => { + beforeAll(async () => { + await import('@/lib/processors/organize-files.processor'); + }, 60000); + + beforeEach(() => { + vi.clearAllMocks(); + root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'rmab-497-org-'))); + organizerHolder.media = path.join(root, 'media'); + organizerHolder.temp = path.join(root, 'tmp'); + source = path.join(root, 'downloads', 'Book'); + target = path.join(organizerHolder.media, 'Author', 'Book'); + fs.mkdirSync(source, { recursive: true }); + fs.mkdirSync(organizerHolder.temp, { recursive: true }); + existingRecord = null; + + prismaMock.request.updateMany.mockResolvedValue({ count: 1 }); + prismaMock.request.findUnique.mockImplementation(async (args: { select?: Record }) => + args?.select?.placedEbookFiles + ? { placedEbookFiles: existingRecord } + : { id: 'req-eb', userId: 'u1', type: 'ebook', user: { plexUsername: 'tester' } } + ); + prismaMock.audiobook.findUnique.mockResolvedValue({ id: 'ab-1', title: 'Book', author: 'Author', narrator: null, audibleAsin: null, year: 2020 }); + prismaMock.downloadHistory.findFirst.mockResolvedValue(null); + prismaMock.downloadHistory.findUnique.mockResolvedValue(null); + prismaMock.configuration.findUnique.mockImplementation(async ({ where }: { where?: { key?: string } }) => { + if (where?.key === 'audiobook_path_template') return { value: '{author}/{title}' }; + if (where?.key === 'media_dir') return { value: organizerHolder.media }; + if (where?.key === 'download_dir') return { value: path.join(root, 'downloads') }; + return null; + }); + configMock.getBackendMode.mockResolvedValue('audiobookshelf'); + configMock.get.mockResolvedValue(null); + }); + + afterEach(() => { + fs.rmSync(root, { recursive: true, force: true }); + }); + + it('copied: a new file in the library is recorded with its path, size and sha256', async () => { + fs.writeFileSync(path.join(source, 'Book.epub'), 'downloaded epub'); + + const result = await runOrganize(false); + + expect(result.success).toBe(true); + expectDelivered(); + const placed = path.join(target, 'Book.epub'); + expect(fs.readFileSync(placed, 'utf8')).toBe('downloaded epub'); + expect(recordWrites()).toEqual([[{ path: placed, size: 15, sha256: sha('downloaded epub'), placedAt: expect.any(String) }]]); + }); + + it.each([ + ['present (same size, not compared)', false, 'user epub bytes', 'downloaded epub'], + ['identical (same bytes, compared)', true, 'downloaded epub', 'downloaded epub'], + ['differs (same size, other bytes, compared)', true, 'user epub bytes', 'downloaded epub'], + ['differs (other size)', false, 'the user\'s own longer edition', 'downloaded epub'], + ])('%s: the library already had the file, so the record is [] and the file is untouched', async (_label, cleanupSource, inLibrary, downloaded) => { + fs.mkdirSync(target, { recursive: true }); + fs.writeFileSync(path.join(target, 'Book.epub'), inLibrary); + fs.writeFileSync(path.join(source, 'Book.epub'), downloaded); + + const result = await runOrganize(cleanupSource); + + expect(result.success).toBe(true); + expectDelivered(); + expect(recordWrites()).toEqual([[]]); + expect(fs.readFileSync(path.join(target, 'Book.epub'), 'utf8')).toBe(inLibrary); + }); + + it('identical in place: a Manual Import of the library copy itself records nothing', async () => { + fs.mkdirSync(target, { recursive: true }); + fs.writeFileSync(path.join(target, 'Book.epub'), 'library epub'); + source = target; + + const result = await runOrganize(true); + + expect(result.success).toBe(true); + expect(recordWrites()).toEqual([[]]); + expect(fs.readFileSync(path.join(target, 'Book.epub'), 'utf8')).toBe('library epub'); + }); + + it('a later import keeps the earlier entries and adds its own; same path: newest wins', async () => { + const earlier = { path: path.join(target, 'Book.pdf'), size: 3, sha256: sha('pdf'), placedAt: '2026-01-01T00:00:00.000Z' }; + const stale = { path: path.join(target, 'Book.epub'), size: 1, sha256: sha('x'), placedAt: '2026-01-01T00:00:00.000Z' }; + existingRecord = [earlier, stale]; + fs.writeFileSync(path.join(source, 'Book.epub'), 'downloaded epub'); + + await runOrganize(false); + + expect(recordWrites()).toEqual([[earlier, { path: path.join(target, 'Book.epub'), size: 15, sha256: sha('downloaded epub'), placedAt: expect.any(String) }]]); + }); + + it('an import that placed nothing keeps the earlier entries as they are', async () => { + const earlier = { path: path.join(target, 'Book.pdf'), size: 3, sha256: sha('pdf'), placedAt: '2026-01-01T00:00:00.000Z' }; + existingRecord = [earlier]; + fs.mkdirSync(target, { recursive: true }); + fs.writeFileSync(path.join(target, 'Book.epub'), 'user epub bytes'); + fs.writeFileSync(path.join(source, 'Book.epub'), 'downloaded epub'); + + await runOrganize(false); + + expect(recordWrites()).toEqual([[earlier]]); + }); + + it('a record write that fails does not fail the import (the file is then never deleted by a request delete)', async () => { + fs.writeFileSync(path.join(source, 'Book.epub'), 'downloaded epub'); + prismaMock.request.update.mockImplementation(async (args: { data: Record }) => { + if ('placedEbookFiles' in args.data) throw new Error('db write failed'); + return {}; + }); + + const result = await runOrganize(false); + + expect(result.success).toBe(true); + expectDelivered(); + expect(fs.existsSync(path.join(target, 'Book.epub'))).toBe(true); + }); +}); diff --git a/tests/services/request-delete-ebook-fixture.ts b/tests/services/request-delete-ebook-fixture.ts new file mode 100644 index 000000000..74e45a4d9 --- /dev/null +++ b/tests/services/request-delete-ebook-fixture.ts @@ -0,0 +1,96 @@ +/** + * Component: Ebook Request Delete Test Fixture (real temp library folders) + * Documentation: documentation/admin-features/request-deletion.md + * + * Shared by the ebook request delete tests: a temp media library, title folders + * as an import leaves them, and placement record entries computed from the file + * on disk (as RMAB stores them in Request.placedEbookFiles). + */ + +import { createHash } from 'crypto'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { generateFilesHash } from '@/lib/utils/files-hash'; + +export interface Library { + root: string; + media: string; + downloads: string; +} + +export function makeLibrary(prefix: string): Library { + const root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), prefix))); + const media = path.join(root, 'media'); + const downloads = path.join(root, 'downloads'); + fs.mkdirSync(media, { recursive: true }); + fs.mkdirSync(downloads, { recursive: true }); + return { root, media, downloads }; +} + +export function write(file: string, content = 'x'): string { + fs.mkdirSync(path.dirname(file), { recursive: true }); + fs.writeFileSync(file, content); + return file; +} + +/** An imported title folder: audio + cover (+ extra files). */ +export function imported(media: string, rel: string, audio = ['Book.m4b'], extra: string[] = []): string { + const dir = path.join(media, rel); + for (const name of [...audio, 'cover.jpg', ...extra]) write(path.join(dir, name), `${name} bytes`); + return dir; +} + +export function book(filePath: string | null, overrides: Record = {}) { + return { + id: 'book-1', + title: 'Book', + author: 'Author', + narrator: null, + audibleAsin: 'B0TEST0001', + plexGuid: null, + absItemId: 'abs-1', + filePath, + fileFormat: 'epub', + filesHash: generateFilesHash(['Book.m4b']), + ...overrides, + }; +} + +/** A placement record entry for `file` as it is on disk now. */ +export function entryFor(file: string) { + const data = fs.readFileSync(file); + return { + path: file, + size: data.length, + sha256: createHash('sha256').update(data).digest('hex'), + placedAt: '2026-10-01T00:00:00.000Z', + }; +} + +/** The ebook request row deleteRequest loads (scalars plus the included book). */ +export function ebookRequest( + audiobook: ReturnType, + fields: { status?: string; completedAt?: Date | null; placedEbookFiles?: unknown } = {} +) { + return { + id: 'req-1', + type: 'ebook', + status: fields.status ?? 'downloaded', + completedAt: fields.completedAt === undefined ? new Date('2026-10-01T00:00:00Z') : fields.completedAt, + placedEbookFiles: fields.placedEbookFiles === undefined ? null : fields.placedEbookFiles, + audiobook, + downloadHistory: [], + }; +} + +/** Every file below `dir` with its bytes (to prove a folder is byte for byte unchanged). */ +export function snapshot(dir: string): Record { + const out: Record = {}; + for (const rel of fs.readdirSync(dir, { recursive: true }) as string[]) { + const full = path.join(dir, rel); + const stat = fs.lstatSync(full); + if (stat.isFile()) out[rel.replace(/\\/g, '/')] = fs.readFileSync(full).toString('hex'); + } + return out; +} diff --git a/tests/services/request-delete.ebook-placement-guard.test.ts b/tests/services/request-delete.ebook-placement-guard.test.ts new file mode 100644 index 000000000..71b810f96 --- /dev/null +++ b/tests/services/request-delete.ebook-placement-guard.test.ts @@ -0,0 +1,260 @@ +/** + * Component: Ebook Request Delete Tests: checks on the recorded file (#497, real filesystem) + * Documentation: documentation/admin-features/request-deletion.md + * + * deleteRequest for an ebook request with a placement record: the recorded file + * goes only when the record reads, rules a-c pass on its folder and the file is + * still exactly what RMAB placed. Reached only through deleteRequest. + */ + +import * as fs from 'fs'; +import * as path from 'path'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { createPrismaMock } from '../helpers/prisma'; +import { book, ebookRequest, entryFor, imported, makeLibrary, snapshot, write, type Library } from './request-delete-ebook-fixture'; + +const prismaMock = createPrismaMock(); +const configServiceMock = { get: vi.fn(), getBackendMode: vi.fn() }; +const unlinkState = vi.hoisted(() => ({ error: null as NodeJS.ErrnoException | null })); + +vi.mock('@/lib/db', () => ({ prisma: prismaMock })); +vi.mock('@/lib/services/config.service', () => ({ getConfigService: () => configServiceMock })); +vi.mock('@/lib/services/audiobookshelf/api', () => ({ getABSItem: vi.fn(), deleteABSItem: vi.fn() })); +vi.mock('fs/promises', async (importOriginal) => { + const actual = await importOriginal(); + const unlink = (async (...args: Parameters) => { + if (unlinkState.error) throw unlinkState.error; + return actual.unlink(...args); + }) as typeof actual.unlink; + return { ...actual, default: { ...actual, unlink }, unlink }; +}); + +const { deleteRequest } = await import('@/lib/services/request-delete.service'); + +const UNREADABLE = 'the placement record could not be read'; + +let lib: Library; +let otherBooks: Array<{ id: string; title: string; filePath: string | null }>; +let downloadDir: string; + +/** A title folder with audio and RMAB's Book.epub, and the request recording that epub. */ +function placed(content = 'rmab epub') { + const dir = imported(lib.media, 'Author/Book'); + const file = write(path.join(dir, 'Book.epub'), content); + return { dir, file, entry: entryFor(file) }; +} + +function load(filePath: string | null, placedEbookFiles: unknown, overrides: Record = {}) { + prismaMock.request.findFirst.mockResolvedValue(ebookRequest(book(filePath, overrides), { placedEbookFiles })); +} + +beforeEach(() => { + vi.clearAllMocks(); + unlinkState.error = null; + lib = makeLibrary('rmab-497-guard-'); + downloadDir = lib.downloads; + otherBooks = []; + prismaMock.configuration.findUnique.mockImplementation(async ({ where }: { where: { key: string } }) => { + if (where.key === 'media_dir') return { value: lib.media }; + if (where.key === 'download_dir') return { value: downloadDir }; + return null; + }); + prismaMock.audiobook.findMany.mockImplementation(async () => otherBooks); + prismaMock.request.findMany.mockResolvedValue([]); + configServiceMock.getBackendMode.mockResolvedValue('audiobookshelf'); + configServiceMock.get.mockResolvedValue(null); +}); + +afterEach(() => { + fs.rmSync(lib.root, { recursive: true, force: true }); +}); + +describe('#497: the recorded file must still be what RMAB placed', () => { + it('keeps a recorded file whose content changed (same size, other bytes)', async () => { + const { dir, file, entry } = placed('rmab epub'); + fs.writeFileSync(file, 'user epub'); + load(dir, [entry]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(false); + expect(result.filesKeptReason).toBe('Book.epub changed since RMAB placed it'); + expect(fs.readFileSync(file, 'utf8')).toBe('user epub'); + }); + + it('keeps a recorded file whose size changed', async () => { + const { dir, file, entry } = placed(); + fs.writeFileSync(file, 'a longer user edition'); + load(dir, [entry]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe('Book.epub changed since RMAB placed it'); + expect(fs.existsSync(file)).toBe(true); + }); + + it('keeps a recorded file replaced by a link (junction/symlink); the link and its target stay', async () => { + const { dir, file, entry } = placed(); + const elsewhere = path.join(lib.root, 'calibre', 'Shelf'); + write(path.join(elsewhere, 'Book.epub'), 'rmab epub'); + fs.rmSync(file); + fs.symlinkSync(elsewhere, file, 'junction'); + load(dir, [entry]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe('Book.epub is no longer the file RMAB placed'); + expect(fs.lstatSync(file).isSymbolicLink()).toBe(true); + expect(fs.existsSync(path.join(elsewhere, 'Book.epub'))).toBe(true); + }); + + it('keeps a recorded path that is now a folder', async () => { + const { dir, file, entry } = placed(); + fs.rmSync(file); + write(path.join(file, 'inside.txt')); + load(dir, [entry]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe('Book.epub is no longer the file RMAB placed'); + expect(fs.existsSync(path.join(file, 'inside.txt'))).toBe(true); + }); + + it('a recorded file that is already gone: nothing removed, no reason, the rest of the folder stays', async () => { + const { dir, file, entry } = placed(); + fs.rmSync(file); + const before = snapshot(lib.media); + load(dir, [entry]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result).toMatchObject({ success: true, filesDeleted: false, message: 'Request deleted successfully' }); + expect(snapshot(lib.media)).toEqual(before); + }); + + it('two entries: removes the intact one, keeps the changed one and says which', async () => { + const { dir, file, entry } = placed(); + const pdf = write(path.join(dir, 'Book.pdf'), 'rmab pdf'); + const pdfEntry = entryFor(pdf); + fs.writeFileSync(pdf, 'user pdf'); + load(dir, [entry, pdfEntry]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(true); + expect(result.filesKeptReason).toBe('Book.pdf changed since RMAB placed it'); + expect(fs.existsSync(file)).toBe(false); + expect(fs.readFileSync(pdf, 'utf8')).toBe('user pdf'); + }); + + it('an unlink failure keeps the file and says why', async () => { + const { dir, file, entry } = placed(); + unlinkState.error = Object.assign(new Error('EACCES: permission denied'), { code: 'EACCES' }); + load(dir, [entry]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe('Book.epub could not be removed (permission denied)'); + expect(fs.existsSync(file)).toBe(true); + }); +}); + +describe('#497: an unreadable record keeps the file', () => { + const sha = 'a'.repeat(64); + it.each([ + ['a relative path', () => [{ path: 'Author/Book/Book.epub', size: 9, sha256: sha }]], + ['a path with a NUL', (file: string) => [{ path: `${file}\0.epub`, size: 9, sha256: sha }]], + ['a non-ebook file', (file: string) => [{ path: path.join(path.dirname(file), 'Book.m4b'), size: 9, sha256: sha }]], + ['a short sha256', (file: string) => [{ ...entryFor(file), sha256: 'abc' }]], + ['an unsafe size', (file: string) => [{ ...entryFor(file), size: Number.MAX_SAFE_INTEGER + 1 }]], + ['a negative size', (file: string) => [{ ...entryFor(file), size: -1 }]], + ['a non-array record', (file: string) => ({ path: file })], + ['an entry that is not an object', () => ['Book.epub']], + ] as Array<[string, (file: string) => unknown]>)('%s: kept with the reason', async (_label, record) => { + const { dir, file } = placed(); + const before = snapshot(lib.media); + load(dir, record(file)); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(false); + expect(result.filesKeptReason).toBe(UNREADABLE); + expect(snapshot(lib.media)).toEqual(before); + }); +}); + +describe('#497: rules a-c on the recorded file\'s own folder', () => { + it('a recorded path outside the media library is kept', async () => { + const outside = write(path.join(lib.root, 'calibre', 'Book.epub'), 'rmab epub'); + load(imported(lib.media, 'Author/Book'), [entryFor(outside)]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toContain('it is outside the media library'); + expect(fs.existsSync(outside)).toBe(true); + }); + + it('a recorded file directly in the media root is kept', async () => { + const atRoot = write(path.join(lib.media, 'Book.epub'), 'rmab epub'); + load(null, [entryFor(atRoot)]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe('it is the media library root'); + expect(fs.existsSync(atRoot)).toBe(true); + }); + + it('a recorded file in a configured root (download folder inside the library) is kept', async () => { + downloadDir = path.join(lib.media, 'incoming'); + const file = write(path.join(downloadDir, 'Book.epub'), 'rmab epub'); + load(null, [entryFor(file)]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toContain('it is a configured root'); + expect(fs.existsSync(file)).toBe(true); + }); + + it('a folder that contains another book\'s folder is kept', async () => { + const authorDir = path.join(lib.media, 'Author'); + const file = write(path.join(authorDir, 'Book.epub'), 'rmab epub'); + const other = imported(lib.media, 'Author/Other Book'); + otherBooks = [{ id: 'book-2', title: 'Other Book', filePath: other }]; + load(authorDir, [entryFor(file)], { filesHash: null }); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe('it contains the folder of "Other Book"'); + expect(fs.existsSync(file)).toBe(true); + }); + + it('a symlinked title folder inside the library: the real file goes, the link stays', async () => { + const real = imported(lib.media, 'Real/Book'); + const file = write(path.join(real, 'Book.epub'), 'rmab epub'); + const link = path.join(lib.media, 'Author', 'Book'); + fs.mkdirSync(path.dirname(link), { recursive: true }); + fs.symlinkSync(real, link, 'junction'); + load(link, [{ ...entryFor(file), path: path.join(link, 'Book.epub') }]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(true); + expect(fs.existsSync(file)).toBe(false); + expect(fs.lstatSync(link).isSymbolicLink()).toBe(true); + expect(fs.readdirSync(real).sort()).toEqual(['Book.m4b', 'cover.jpg']); + }); + + it('a symlinked title folder pointing outside the library is kept; the target is intact', async () => { + const seeding = path.join(lib.downloads, 'Book [seed]'); + const file = write(path.join(seeding, 'Book.epub'), 'rmab epub'); + const link = path.join(lib.media, 'Author', 'Book'); + fs.mkdirSync(path.dirname(link), { recursive: true }); + fs.symlinkSync(seeding, link, 'junction'); + load(link, [{ ...entryFor(file), path: path.join(link, 'Book.epub') }]); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toContain('outside the media library'); + expect(fs.existsSync(file)).toBe(true); + }); +}); diff --git a/tests/services/request-delete.ebook-placement.test.ts b/tests/services/request-delete.ebook-placement.test.ts new file mode 100644 index 000000000..bb0750894 --- /dev/null +++ b/tests/services/request-delete.ebook-placement.test.ts @@ -0,0 +1,220 @@ +/** + * Component: Ebook Request Delete Tests: only the file that request placed (#497, real filesystem) + * Documentation: documentation/admin-features/request-deletion.md + * + * deleteRequest for ebook requests with real temp folders. The request's own + * placement record (Request.placedEbookFiles) decides what goes, not its status + * and not what else sits in the book folder. Reached only through deleteRequest. + */ + +import * as fs from 'fs'; +import * as path from 'path'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { createPrismaMock } from '../helpers/prisma'; +import { book, ebookRequest, entryFor, imported, makeLibrary, snapshot, write, type Library } from './request-delete-ebook-fixture'; + +const prismaMock = createPrismaMock(); +const configServiceMock = { get: vi.fn(), getBackendMode: vi.fn() }; +const absMock = vi.hoisted(() => ({ getABSItem: vi.fn(), deleteABSItem: vi.fn() })); + +vi.mock('@/lib/db', () => ({ prisma: prismaMock })); +vi.mock('@/lib/services/config.service', () => ({ getConfigService: () => configServiceMock })); +vi.mock('@/lib/services/audiobookshelf/api', () => absMock); + +const { deleteRequest } = await import('@/lib/services/request-delete.service'); + +const LEGACY = 'this request was imported before RMAB kept a record of the ebook files it places, so no ebook file was removed'; +const ALREADY_THERE = 'the ebook was already in the library when this request imported it, so it was not removed'; + +let lib: Library; +let otherBooks: Array<{ id: string; title: string; filePath: string | null }>; +/** Other active requests' placement records (the step 2 lookup). */ +let otherRecords: Array<{ placedEbookFiles: unknown }>; + +function load(req: ReturnType) { + prismaMock.request.findFirst.mockResolvedValue(req); +} + +/** No backend delete, no record cleanup, no book row change; the request is soft-deleted. */ +function expectLibraryRowsUntouched() { + expect(absMock.deleteABSItem).not.toHaveBeenCalled(); + expect(prismaMock.plexLibrary.deleteMany).not.toHaveBeenCalled(); + expect(prismaMock.audiobook.update).not.toHaveBeenCalled(); + expect(prismaMock.request.update).toHaveBeenCalledWith(expect.objectContaining({ data: expect.objectContaining({ deletedBy: 'admin-1' }) })); +} + +beforeEach(() => { + vi.clearAllMocks(); + lib = makeLibrary('rmab-497-delete-'); + otherBooks = []; + otherRecords = []; + prismaMock.configuration.findUnique.mockImplementation(async ({ where }: { where: { key: string } }) => { + if (where.key === 'media_dir') return { value: lib.media }; + if (where.key === 'download_dir') return { value: lib.downloads }; + return null; + }); + prismaMock.audiobook.findMany.mockImplementation(async () => otherBooks); + prismaMock.request.findMany.mockImplementation(async () => otherRecords); + configServiceMock.getBackendMode.mockResolvedValue('audiobookshelf'); + configServiceMock.get.mockResolvedValue(null); +}); + +afterEach(() => { + fs.rmSync(lib.root, { recursive: true, force: true }); +}); + +describe('#497: an ebook request delete removes only the ebook file that request placed', () => { + it('issue steps: imported audiobook + the user\'s own Book.epub + a failed ebook request: the epub stays, no reason', async () => { + const dir = imported(lib.media, 'Author/Book B0TEST0001', ['Book.m4b'], ['Book.epub']); + const before = snapshot(lib.media); + load(ebookRequest(book(dir), { status: 'failed', completedAt: null })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result).toMatchObject({ success: true, filesDeleted: false, message: 'Request deleted successfully' }); + expect(result.filesKeptReason).toBeUndefined(); + expect(snapshot(lib.media)).toEqual(before); + expectLibraryRowsUntouched(); + }); + + it.each([ + 'pending', 'awaiting_approval', 'denied', 'searching', 'downloading', 'processing', + 'awaiting_search', 'awaiting_release', 'awaiting_import', 'failed', 'warn', 'cancelled', 'available', + ])('a %s ebook request with no record deletes no file (the only ebook in the folder stays)', async (status) => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub']); + const before = snapshot(lib.media); + load(ebookRequest(book(dir), { status, completedAt: null })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.success).toBe(true); + expect(result.filesDeleted).toBe(false); + expect(result.filesKeptReason).toBeUndefined(); + expect(snapshot(lib.media)).toEqual(before); + }); + + it('a pre-upgrade delivered request (no record, NULL) keeps the ebook and says it was imported before RMAB kept records', async () => { + const dir = path.join(lib.media, 'Ebooks'); + write(path.join(dir, 'Book.epub')); + load(ebookRequest(book(dir, { filesHash: null }), { status: 'downloaded', placedEbookFiles: null })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(false); + expect(result.filesKeptReason).toBe(LEGACY); + expect(result.message).toBe(`Request deleted; files kept: ${LEGACY}`); + expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(true); + expectLibraryRowsUntouched(); + }); + + it('a request recycled after delivery (completedAt set, now failed) with no record also gets the pre-upgrade reason', async () => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub']); + load(ebookRequest(book(dir), { status: 'failed', placedEbookFiles: null })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe(LEGACY); + expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(true); + }); + + it('a delivered request whose import placed nothing ([]: the ebook was already there) keeps it and says so', async () => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub']); + const before = snapshot(lib.media); + load(ebookRequest(book(dir), { status: 'downloaded', placedEbookFiles: [] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(false); + expect(result.filesKeptReason).toBe(ALREADY_THERE); + expect(snapshot(lib.media)).toEqual(before); + }); + + it('removes the recorded epub; the user\'s azw3, the audio and the cover stay; no backend or row changes', async () => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub', 'Book.azw3']); + load(ebookRequest(book(dir), { placedEbookFiles: [entryFor(path.join(dir, 'Book.epub'))] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result).toMatchObject({ success: true, filesDeleted: true, message: 'Request deleted successfully' }); + expect(result.filesKeptReason).toBeUndefined(); + expect(fs.readdirSync(dir).sort()).toEqual(['Book.azw3', 'Book.m4b', 'cover.jpg']); + expectLibraryRowsUntouched(); + }); + + it('a flat folder with the recorded ebook and foreign ebooks: only the recorded one goes', async () => { + const dir = path.join(lib.media, 'Ebooks'); + write(path.join(dir, 'Book.epub'), 'rmab copy'); + write(path.join(dir, 'Calibre Book.epub')); + load(ebookRequest(book(dir, { filesHash: null }), { placedEbookFiles: [entryFor(path.join(dir, 'Book.epub'))] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(true); + expect(fs.readdirSync(dir)).toEqual(['Calibre Book.epub']); + }); + + it('second ebook request for the same book in another format: deleting A removes only A\'s epub', async () => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub', 'Book.azw3']); + otherRecords = [{ placedEbookFiles: [entryFor(path.join(dir, 'Book.azw3'))] }]; + load(ebookRequest(book(dir), { placedEbookFiles: [entryFor(path.join(dir, 'Book.epub'))] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(true); + expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(false); + expect(fs.existsSync(path.join(dir, 'Book.azw3'))).toBe(true); + }); + + it('keeps a file another active request also recorded, and looks at every active request (not only this book\'s)', async () => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub']); + const entry = entryFor(path.join(dir, 'Book.epub')); + otherRecords = [{ placedEbookFiles: [entry] }]; + load(ebookRequest(book(dir), { placedEbookFiles: [entry] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(false); + expect(result.filesKeptReason).toBe('Book.epub is also the ebook of another request'); + expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(true); + const where = prismaMock.request.findMany.mock.calls[0][0].where; + expect(where).toMatchObject({ id: { not: 'req-1' }, deletedAt: null }); + expect(where.audiobookId).toBeUndefined(); + }); + + it('a recycled request (Manual Import failed after an earlier delivery) still removes the file its earlier import placed', async () => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub']); + load(ebookRequest(book(dir), { status: 'failed', placedEbookFiles: [entryFor(path.join(dir, 'Book.epub'))] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(true); + expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(false); + expect(fs.existsSync(path.join(dir, 'Book.m4b'))).toBe(true); + }); + + it('custom ebook_path_template: the recorded file in its own folder is removed even though filePath is the audio folder', async () => { + const audioDir = imported(lib.media, 'Author/Book'); + const ebook = write(path.join(lib.media, 'Ebooks', 'Author', 'Book.epub'), 'rmab ebook'); + write(path.join(lib.media, 'Ebooks', 'Author', 'Other.epub')); + load(ebookRequest(book(audioDir), { placedEbookFiles: [entryFor(ebook)] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(true); + expect(fs.existsSync(ebook)).toBe(false); + expect(fs.existsSync(path.join(lib.media, 'Ebooks', 'Author', 'Other.epub'))).toBe(true); + expect(fs.readdirSync(audioDir).sort()).toEqual(['Book.m4b', 'cover.jpg']); + }); + + it('a recorded file in a flat folder another book points at is kept (rule c), with the reason', async () => { + const dir = path.join(lib.media, 'Ebooks'); + write(path.join(dir, 'Book.epub')); + otherBooks = [{ id: 'book-2', title: 'Second', filePath: dir }]; + load(ebookRequest(book(dir, { filesHash: null }), { placedEbookFiles: [entryFor(path.join(dir, 'Book.epub'))] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe('it is also the folder of "Second"'); + expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(true); + }); +}); diff --git a/tests/services/request-delete.library-guard.test.ts b/tests/services/request-delete.library-guard.test.ts index cbb89104e..145b9b1a1 100644 --- a/tests/services/request-delete.library-guard.test.ts +++ b/tests/services/request-delete.library-guard.test.ts @@ -361,53 +361,3 @@ describe('deleteRequest (audiobook): stored folder behind the guard', () => { }); }); }); - -describe('deleteRequest (ebook): ebook files behind the guard', () => { - it('removes the ebooks in a folder that passes the evidence rule; audio stays; no backend or row changes', async () => { - const dir = imported('Author/Book', ['Book.m4b'], ['Book.epub']); - request(book(dir), 'ebook'); - - const result = await deleteRequest('req-1', 'admin-1'); - - expect(result.filesDeleted).toBe(true); - expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(false); - expect(fs.existsSync(path.join(dir, 'Book.m4b'))).toBe(true); - expectLibraryUntouched(); - }); - - it('removes the only ebook in a flat Ebooks folder', async () => { - const dir = path.join(media, 'Ebooks'); - write(path.join(dir, 'Book.epub')); - request(book(dir, { filesHash: null }), 'ebook'); - - const result = await deleteRequest('req-1', 'admin-1'); - - expect(result.filesDeleted).toBe(true); - expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(false); - }); - - it('keeps a flat folder holding foreign ebooks besides this one, and says why', async () => { - const dir = path.join(media, 'Ebooks'); - write(path.join(dir, 'Book.epub')); - write(path.join(dir, 'Calibre Book.epub')); - request(book(dir, { filesHash: null }), 'ebook'); - - const result = await deleteRequest('req-1', 'admin-1'); - - expect(result.filesDeleted).toBe(false); - expect(result.filesKeptReason).toContain('holds 2 ebook files'); - expect(fs.readdirSync(dir).sort()).toEqual(['Book.epub', 'Calibre Book.epub']); - }); - - it('keeps a folder another book also points at', async () => { - const dir = path.join(media, 'Ebooks'); - write(path.join(dir, 'Book.epub')); - otherBooks = [{ id: 'book-2', title: 'Second', filePath: dir }]; - request(book(dir, { filesHash: null }), 'ebook'); - - const result = await deleteRequest('req-1', 'admin-1'); - - expect(result.filesKeptReason).toBe('it is also the folder of "Second"'); - expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(true); - }); -}); diff --git a/tests/services/request-delete.service.test.ts b/tests/services/request-delete.service.test.ts index b90bf6374..3d7650cac 100644 --- a/tests/services/request-delete.service.test.ts +++ b/tests/services/request-delete.service.test.ts @@ -40,7 +40,7 @@ vi.mock('@/lib/services/audiobookshelf/api', () => ({ })); // The library delete guard has its own real-filesystem tests; here it allows every folder -const guardMock = vi.hoisted(() => ({ assertSafeLibraryDelete: vi.fn(), planEbookDelete: vi.fn(), isStaleStoredFolder: vi.fn(async () => false) })); +const guardMock = vi.hoisted(() => ({ assertSafeLibraryDelete: vi.fn(), isStaleStoredFolder: vi.fn(async () => false) })); vi.mock('@/lib/utils/library-delete-guard', () => guardMock); vi.mock('@/lib/processors/organize-cleanup-guard', () => ({ sweepEmptyParents: vi.fn(async () => undefined) })); const canonical = (p: string) => path.resolve(p).replace(/\\/g, '/'); diff --git a/tests/utils/ebook-placement.test.ts b/tests/utils/ebook-placement.test.ts new file mode 100644 index 000000000..08bbfe813 --- /dev/null +++ b/tests/utils/ebook-placement.test.ts @@ -0,0 +1,173 @@ +/** + * Component: Ebook Placement Record Tests (#497) + * Documentation: documentation/admin-features/request-deletion.md + * + * Reading, building and merging Request.placedEbookFiles, and the record write + * (only a 'copied' placement adds an entry; it never throws). + */ + +import { createHash } from 'crypto'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { createPrismaMock } from '../helpers/prisma'; + +const prismaMock = createPrismaMock(); +vi.mock('@/lib/db', () => ({ prisma: prismaMock })); + +const { mergePlacements, placementEntry, readPlacements, recordEbookPlacement, otherRecordedPaths } = await import('@/lib/utils/ebook-placement'); + +const sha = (data: string) => createHash('sha256').update(data).digest('hex'); +const logger = { info: vi.fn(), warn: vi.fn() } as unknown as Parameters[2]; + +let root: string; + +beforeEach(() => { + vi.clearAllMocks(); + root = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'rmab-497-entry-'))); +}); + +afterEach(() => { + fs.rmSync(root, { recursive: true, force: true }); +}); + +/** An absolute library folder for record values (nothing is read from it). */ +const LIB = path.resolve('/library/Author/Book'); + +describe('readPlacements', () => { + const good = () => ({ path: path.join(LIB, 'Book.epub'), size: 5, sha256: 'f'.repeat(64), placedAt: '2026-10-01T00:00:00.000Z' }); + + it('null and undefined are "no record" (SQL NULL and JSON null read the same)', () => { + expect(readPlacements(null)).toBeNull(); + expect(readPlacements(undefined)).toBeNull(); + }); + + it('[] is a record with no entries', () => { + expect(readPlacements([])).toEqual({ entries: [], unreadable: 0 }); + }); + + it('keeps well-formed entries and counts the rest', () => { + const ok = good(); + expect(readPlacements([ok, { ...ok, path: 'relative/Book.epub' }, 'x', null])).toEqual({ entries: [ok], unreadable: 3 }); + }); + + it.each([ + ['a relative path', { path: 'Book.epub' }], + ['a NUL in the path', { path: `${path.join(LIB, 'Book')}\0.epub` }], + ['a non-ebook file', { path: path.join(LIB, 'Book.m4b') }], + ['an upper-case sha256', { sha256: 'F'.repeat(64) }], + ['a 63-char sha256', { sha256: 'f'.repeat(63) }], + ['a fractional size', { size: 1.5 }], + ['a string size', { size: '5' }], + ['a negative size', { size: -1 }], + ['an unsafe size', { size: 2 ** 53 }], + ])('rejects %s', (_label, change) => { + expect(readPlacements([{ ...good(), ...change }])).toEqual({ entries: [], unreadable: 1 }); + }); + + it('accepts every organizer ebook format, any case', () => { + for (const name of ['B.epub', 'B.PDF', 'B.azw3', 'B.mobi']) { + expect(readPlacements([{ ...good(), path: path.join(LIB, name) }])?.entries).toHaveLength(1); + } + }); + + it('a non-array value is one unreadable item', () => { + expect(readPlacements({ path: '/x.epub' })).toEqual({ entries: [], unreadable: 1 }); + }); +}); + +describe('placementEntry', () => { + it('reads the size and sha256 of the placed file', async () => { + const file = path.join(root, 'Book.epub'); + fs.writeFileSync(file, 'epub bytes'); + expect(await placementEntry(file)).toEqual({ path: file, size: 10, sha256: sha('epub bytes'), placedAt: expect.any(String) }); + }); + + it('refuses anything that is not a regular file', async () => { + fs.mkdirSync(path.join(root, 'Book.epub')); + await expect(placementEntry(path.join(root, 'Book.epub'))).rejects.toThrow('not a regular file'); + }); +}); + +describe('mergePlacements', () => { + const a = { path: '/m/A.epub', size: 1, sha256: 'a'.repeat(64), placedAt: 't1' }; + const b = { path: '/m/B.epub', size: 1, sha256: 'b'.repeat(64), placedAt: 't1' }; + + it('starts a record from NULL ([] when nothing was placed)', () => { + expect(mergePlacements(null, null)).toEqual([]); + expect(mergePlacements(null, a)).toEqual([a]); + }); + + it('appends; the same path is replaced by the newest entry', () => { + const newer = { ...a, sha256: 'c'.repeat(64), placedAt: 't2' }; + expect(mergePlacements([a, b], newer)).toEqual([b, newer]); + }); + + it('keeps whatever was stored, readable or not', () => { + expect(mergePlacements([a, 'junk'], b)).toEqual([a, 'junk', b]); + expect(mergePlacements({ odd: true }, b)).toEqual([{ odd: true }, b]); + }); +}); + +describe('recordEbookPlacement', () => { + function placed(status: string) { + const file = path.join(root, 'Book.epub'); + fs.writeFileSync(file, 'placed'); + return { source: '/dl/Book.epub', target: file, status, verified: true } as Parameters[1]; + } + + it('copied: writes the entry, merged into the stored record', async () => { + const earlier = { path: '/m/A.pdf', size: 1, sha256: 'a'.repeat(64), placedAt: 't1' }; + prismaMock.request.findUnique.mockResolvedValue({ placedEbookFiles: [earlier] }); + const record = placed('copied'); + + await recordEbookPlacement('req-1', record, logger); + + expect(prismaMock.request.findUnique).toHaveBeenCalledWith({ where: { id: 'req-1' }, select: { placedEbookFiles: true } }); + expect(prismaMock.request.update).toHaveBeenCalledWith({ + where: { id: 'req-1' }, + data: { placedEbookFiles: [earlier, { path: record!.target, size: 6, sha256: sha('placed'), placedAt: expect.any(String) }] }, + }); + }); + + it.each(['identical', 'present', 'differs'])('%s: writes the stored record unchanged ([] from NULL), no entry', async (status) => { + prismaMock.request.findUnique.mockResolvedValue({ placedEbookFiles: null }); + + await recordEbookPlacement('req-1', placed(status), logger); + + expect(prismaMock.request.update).toHaveBeenCalledWith({ where: { id: 'req-1' }, data: { placedEbookFiles: [] } }); + }); + + it('never throws: a database failure is logged and the import goes on', async () => { + prismaMock.request.findUnique.mockRejectedValue(new Error('db down')); + + await expect(recordEbookPlacement('req-1', placed('copied'), logger)).resolves.toBeUndefined(); + expect(prismaMock.request.update).not.toHaveBeenCalled(); + expect(logger.warn).toHaveBeenCalledWith(expect.stringContaining('deleting this request will keep it')); + }); + + it('never throws: a placed file that cannot be read leaves the field as it was', async () => { + const record = { source: '/dl/x.epub', target: path.join(root, 'missing.epub'), status: 'copied', verified: true } as Parameters[1]; + + await recordEbookPlacement('req-1', record, logger); + + expect(prismaMock.request.update).not.toHaveBeenCalled(); + expect(logger.warn).toHaveBeenCalled(); + }); +}); + +describe('otherRecordedPaths', () => { + it('collects readable entries of other active ebook requests, of any book', async () => { + const entry = { path: path.join(root, 'A.epub'), size: 1, sha256: 'a'.repeat(64), placedAt: 't' }; + prismaMock.request.findMany.mockResolvedValue([{ placedEbookFiles: [entry, 'junk'] }, { placedEbookFiles: null }]); + + const paths = await otherRecordedPaths('req-1'); + + expect([...paths]).toEqual([path.resolve(entry.path)]); + expect(prismaMock.request.findMany).toHaveBeenCalledWith({ + where: { id: { not: 'req-1' }, deletedAt: null, type: 'ebook' }, + select: { placedEbookFiles: true }, + }); + }); +}); diff --git a/tests/utils/library-delete-guard.test.ts b/tests/utils/library-delete-guard.test.ts index 8e3051b02..edd9ffbf0 100644 --- a/tests/utils/library-delete-guard.test.ts +++ b/tests/utils/library-delete-guard.test.ts @@ -13,7 +13,7 @@ import { generateFilesHash } from '@/lib/utils/files-hash'; const prismaMock = createPrismaMock(); vi.mock('@/lib/db', () => ({ prisma: prismaMock })); -const { assertSafeLibraryDelete, isStaleStoredFolder, planEbookDelete } = await import('@/lib/utils/library-delete-guard'); +const { assertSafeLibraryDelete, isStaleStoredFolder } = await import('@/lib/utils/library-delete-guard'); let root: string; let media: string; @@ -194,45 +194,6 @@ describe('assertSafeLibraryDelete', () => { }); }); -describe('planEbookDelete', () => { - it('returns every ebook directly in a folder that passes the evidence rule', async () => { - const { dir, book } = titleFolder('Author/Book'); - write(path.join(dir, 'Book.epub')); - write(path.join(dir, 'Book.pdf')); - const files = await planEbookDelete(dir, book); - expect(files.map((f) => path.basename(f)).sort()).toEqual(['Book.epub', 'Book.pdf']); - }); - - it('returns the only ebook in a folder without audio evidence (ebook-only book)', async () => { - const dir = path.join(media, 'Ebooks'); - write(path.join(dir, 'Mine.epub')); - const files = await planEbookDelete(dir, { id: 'b', filePath: dir, filesHash: null }); - expect(files.map((f) => path.basename(f))).toEqual(['Mine.epub']); - }); - - it('refuses a flat folder with several ebooks that is not provably this book\'s', async () => { - const dir = path.join(media, 'Ebooks'); - write(path.join(dir, 'Mine.epub')); - write(path.join(dir, 'Calibre Book.epub')); - const reason = await refusal(planEbookDelete(dir, { id: 'b', filePath: dir, filesHash: null })); - expect(reason).toContain('holds 2 ebook files'); - expect(fs.existsSync(path.join(dir, 'Calibre Book.epub'))).toBe(true); - }); - - it('refuses a folder shared with another book (rule c), even with one ebook', async () => { - const dir = path.join(media, 'Ebooks'); - write(path.join(dir, 'Mine.epub')); - otherBooks = [{ id: 'book-2', title: 'Second', filePath: dir }]; - expect(await refusal(planEbookDelete(dir, { id: 'b', filePath: dir, filesHash: null }))).toBe('it is also the folder of "Second"'); - }); - - it('returns nothing for a missing folder (library mounted)', async () => { - write(path.join(media, 'Other', 'x.epub')); - const dir = path.join(media, 'Author', 'Gone'); - await expect(planEbookDelete(dir, { id: 'b', filePath: dir, filesHash: null })).resolves.toEqual([]); - }); -}); - describe('isStaleStoredFolder', () => { it('is true only for a folder that is gone AND outside the current media_dir (library moved)', async () => { expect(await isStaleStoredFolder(path.join(root, 'old-library', 'Author', 'Book'))).toBe(true); From 4af2ad7d5e1e80a2f8c99ec9d2c593465a21809a Mon Sep 17 00:00:00 2001 From: kikootwo Date: Mon, 2 Nov 2026 10:11:07 -0500 Subject: [PATCH 2/3] fix: request responses leave out the ebook placement record; shared-file check and kept reasons - No response that returns a request row carries placedEbookFiles (absolute library paths): GET/POST /api/requests, GET/PATCH /api/requests/[id], manual-search, select-torrent, request-with-torrent, admin approve, pending-approval and the reported-issue replace. - "Another request recorded this file" now means the same file: same sha256 and size, and the other entry's path reaches the same file (dev/ino), so a symlinked or differently spelled folder counts and a stale entry does not. - A request that delivered before its first recorded import gets an earlier-import marker; its delete still removes the recorded files and says the earlier file was kept. Kept reasons no longer claim a pre-upgrade import when the record write failed, and each reason shows once. Co-Authored-By: Claude Opus 5.5 --- .../admin-features/request-deletion.md | 19 ++- documentation/backend/database.md | 2 +- documentation/backend/services/api-tokens.md | 2 +- documentation/phase3/file-organization.md | 2 +- documentation/testing/e2e-harness.md | 4 +- .../regressions/497-ebook-legacy.spec.ts | 2 +- e2e/specs/regressions/497-helpers.ts | 4 +- .../reported-issues/[id]/replace/route.ts | 3 +- .../api/admin/requests/[id]/approve/route.ts | 7 +- .../admin/requests/pending-approval/route.ts | 3 +- .../api/requests/[id]/manual-search/route.ts | 3 +- src/app/api/requests/[id]/route.ts | 8 +- src/app/api/requests/route.ts | 6 +- src/lib/services/request-delete-files.ts | 21 ++- src/lib/services/request-detail-query.ts | 17 ++- src/lib/utils/ebook-placement.ts | 79 +++++++--- .../requests-placement-record.routes.test.ts | 143 ++++++++++++++++++ .../request-delete.ebook-placement.test.ts | 83 +++++++++- tests/utils/ebook-placement.test.ts | 50 ++++-- 19 files changed, 388 insertions(+), 70 deletions(-) create mode 100644 tests/api/requests-placement-record.routes.test.ts diff --git a/documentation/admin-features/request-deletion.md b/documentation/admin-features/request-deletion.md index dbf1e6396..626c527d8 100644 --- a/documentation/admin-features/request-deletion.md +++ b/documentation/admin-features/request-deletion.md @@ -124,18 +124,19 @@ model Request { - Stale record: the stored folder is gone AND outside the current `media_dir` (library moved): nothing removed or protected; the delete clears only `filePath`/`filesHash`/`fileFormat` (no backend call, records and status kept; message says so), a replace goes on - `rm` failure: kept (`the folder was partly removed (permission denied); remove the rest manually`); `fs.rm` stops at the first error, so earlier files are gone; backend and row untouched - After removal: empty parents swept up to the nearest root (`sweepEmptyParents`) - - **Ebook requests (D2, #497):** only the files the request's imports recorded in `Request.placedEbookFiles` (`[{ path, size, sha256, placedAt }]`, `src/lib/utils/ebook-placement.ts`); status and the rest of the folder never count. `Audiobook.filePath` is not read. Per entry (all kept reasons joined `; `): + - **Ebook requests (D2, #497):** only the files the request's imports recorded in `Request.placedEbookFiles` (`[{ path, size, sha256, placedAt }]`, `src/lib/utils/ebook-placement.ts`); status and the rest of the folder never count. `Audiobook.filePath` is not read. Per entry, in order (kept reasons joined `; `, each reason once): | Check | Kept reason | |-------|-------------| | entry well-formed: absolute path, no NUL, ebook extension (`isEbookFile`), safe-integer size ≥ 0, 64-hex sha256 (non-array value = one bad entry) | `the placement record could not be read` | - | path not recorded by another active (`deletedAt null`) ebook request, any book | ` is also the ebook of another request` | | rules a-c on the file's folder (`assertSafeLibraryDelete(dir, book, { evidence: false })`; symlinked folder: target + link location) | rule a-c reasons above | | `lstat` regular file (not a symlink/folder) | ` is no longer the file RMAB placed` | | size and sha256 match the entry | ` changed since RMAB placed it` | + | not the same file as an entry of another active (`deletedAt null`) ebook request, any book: same sha256 + size and its path reaches the same file (`lstat` dev/ino, so a symlinked folder or another spelling of the folder counts; a stale entry with other bytes does not) | ` is also the ebook of another request` (that lstat fails other than ENOENT/ENOTDIR: ` could not be compared with another request's record ()`) | | `unlink` | ` could not be removed ()` | - File or its folder already gone (library mounted): nothing, no reason - - No record: SQL NULL or JSON null (imported before the field, or never imported). Kept reason only when the request ever delivered (`status = 'downloaded'` or `completedAt`): `this request was imported before RMAB kept a record of the ebook files it places, so no ebook file was removed`; otherwise none (`Request deleted successfully`) - - `[]` (the import found the file already in the library: `identical`/`present`/`differs`): `the ebook was already in the library when this request imported it, so it was not removed` + - No record: SQL NULL or JSON null (imported before the field, a record write that failed, or never imported). Kept reason only when the request ever delivered (`status = 'downloaded'` or `completedAt`): `RMAB has no record of the ebook files this request placed (it was imported before RMAB kept one, or the record could not be saved), so no ebook file was removed`; otherwise none (`Request deleted successfully`) + - `[]` (the import found the file already in the library: `identical`/`present`/`differs`): `RMAB has no record that this request placed the ebook (its import found it already in the library), so it was not removed` + - Earlier-import marker `{ "earlierImport": "unrecorded" }` (first item; `recordEbookPlacement` writes it when the stored value is NULL but `completedAt` is already set, i.e. the request delivered before: a pre-upgrade import or a failed record write, then a re-import such as Manual Import): never removable; adds `an earlier import of this request has no record (it was imported before RMAB kept one, or the record could not be saved), so its ebook file was not removed`; the recorded entries after it are still removed - Never a folder sweep: a flat `Ebooks`/shelf folder loses only the recorded file (foreign ebooks stay); a folder another book stores still fails rule c - Not handled (needs a separate ebook location, PR2): an ebook in a separate folder (custom `ebook_path_template`) is not removed by the audiobook delete @@ -323,7 +324,15 @@ where: { - **Issue:** every ebook request delete (any status, e.g. a `failed` one that never downloaded) unlinked the only ebook in `Audiobook.filePath`, or every ebook there when the folder passed the audio evidence rule: the user's own epub beside an imported audiobook went - **Cause:** nothing recorded which file an ebook import placed; `planEbookDelete` guessed from the folder - **Fix:** the ebook organize records the file it copied (`Request.placedEbookFiles`, only for placeFile `copied`); the delete removes only recorded, unchanged files behind rules a-c (D2 above). `planEbookDelete` removed -- **Limit:** ebook requests imported before this version have no record: their delete removes no ebook file (reason shown; remove by hand). A crash between the copy and the record write leaves that file unrecorded (kept) +- **Responses:** the record holds absolute library paths and is read only by the delete: every response that returns a request row leaves it out, for admins too (`hidePlacementRecord()`/`hideStoredPick()` in `src/lib/services/request-detail-query.ts`; `GET /api/requests` sets it `undefined` as it does `audiobook.filePath`) +- **Limits** (each keeps a file; none removes more): + - Ebook requests imported before this version have no record: their delete removes no ebook file (reason shown; remove by hand). A crash between the copy and the record write, or a failed record write, leaves that file unrecorded (kept) + - Another active ebook request on the same book whose import found this file already there (`[]`) does not hold it: deleting the request that placed it removes it, and the other request still shows `downloaded` (as before #497) + - A delete while the request's import is still running: the file that import places afterwards is not recorded on a live request, so RMAB never removes it + - Two organizes of one request at once (e.g. a Manual Import while a stalled job re-runs) can lose one entry: that file is kept + - A flat ebook shelf stored by two books (`ebook_path_template = Ebooks`): rule c keeps every recorded file there (`it is also the folder of ""`) + - After `media_dir` moves, recorded paths under the old location fail rule a (`it is outside the media library (...)`) and are kept + - The ebook's own folder (custom `ebook_path_template`) is left empty after its only file goes (no empty-parent sweep) ## File Structure diff --git a/documentation/backend/database.md b/documentation/backend/database.md index 6d719ed67..bcf49145b 100644 --- a/documentation/backend/database.md +++ b/documentation/backend/database.md @@ -72,7 +72,7 @@ PostgreSQL database storing users, audiobooks, requests, downloads, configuratio - `search_attempts`, `download_attempts`, `import_attempts`, `max_import_retries` (default 5) - `auto_rerequest_count` (Int, default 0) - Automatic re-requests after a failure with no new block (shelf sync / watched lists backoff step); user requests and retries reset it to 0. See [release-blocklist.md](../admin-features/release-blocklist.md#automatic-re-requests-409) - `last_search_at`, `last_import_at`, `created_at`, `updated_at`, `completed_at` -- `placed_ebook_files` (Json, nullable) - Ebook requests: files this request's imports copied into the library, `[{ path, size, sha256, placedAt }]`; `[]` = imported but the file was already there; NULL = no record (pre-#497 or never imported). The request delete removes only these. See [request-deletion.md](../admin-features/request-deletion.md) +- `placed_ebook_files` (Json, nullable) - Ebook requests: files this request's imports copied into the library, `[{ path, size, sha256, placedAt }]`; `[]` = imported but the file was already there; NULL = no record (pre-#497, a failed record write, or never imported); a first item `{ earlierImport: 'unrecorded' }` = the request delivered before its first recorded import. The request delete removes only these. Never in an API response. See [request-deletion.md](../admin-features/request-deletion.md) - Unique: `(user_id, audiobook_id)` - Indexes: `user_id`, `audiobook_id`, `status`, `created_at DESC` diff --git a/documentation/backend/services/api-tokens.md b/documentation/backend/services/api-tokens.md index dd143fd31..e98ef2483 100644 --- a/documentation/backend/services/api-tokens.md +++ b/documentation/backend/services/api-tokens.md @@ -46,7 +46,7 @@ Source of truth: `src/lib/constants/api-tokens.ts` (`API_TOKEN_ALLOWED_ENDPOINTS ## GET `/api/requests/:id` - Returns request including `audiobook`, `downloadHistory` (selected), and recent `jobs`. -- Non-admin callers (owner sessions, user-role tokens): explicit field list `OWNER_REQUEST_DETAIL_SELECT` (`src/lib/services/request-detail-query.ts`) — omits `Request.selectedTorrent`, `DownloadHistory.magnetLink/torrentUrl/downloadPath`, `Audiobook.filePath`, `Job.payload/result/stackTrace`; remaining strings pass through `stripProwlarrKeyFromResults()`. Admins get the full include. +- Non-admin callers (owner sessions, user-role tokens): explicit field list `OWNER_REQUEST_DETAIL_SELECT` (`src/lib/services/request-detail-query.ts`) — omits `Request.selectedTorrent`, `DownloadHistory.magnetLink/torrentUrl/downloadPath`, `Audiobook.filePath`, `Job.payload/result/stackTrace`; remaining strings pass through `stripProwlarrKeyFromResults()`. Admins get the full include, minus `Request.placedEbookFiles` (never returned by any request response: `GET /api/requests` list, `POST /api/requests` and this route included). - Ownership enforced: `requestRecord.userId === req.user.id || role === 'admin'` → otherwise 403. - Soft-deleted requests (`deletedAt != null`) return 404. - BigInt columns (`downloadHistory[].torrentSizeBytes`, `audiobook.fileSizeBytes`) go out as JSON numbers via `toJsonSafe()` (`src/lib/utils/json-safe.ts`). diff --git a/documentation/phase3/file-organization.md b/documentation/phase3/file-organization.md index 5e98f913d..48a1b651f 100644 --- a/documentation/phase3/file-organization.md +++ b/documentation/phase3/file-organization.md @@ -343,7 +343,7 @@ When user deletes and re-requests the same audiobook: - Permissions (`applyFileMode` → `applied` / `refused` / `missing`): after a copy is published, `chmod(file_chmod)` is best-effort. mergerfs/NFS/FUSE/SMB often refuse it (EPERM, ENOTSUP, EACCES, …) though the write succeeded → placement stays `copied` with `PlaceResult.warning` (path, octal mode, code). ENOENT/ENOTDIR mean no file at the path → `failed`, never verified. Same for downloaded/cached cover art (missing = cover failed). No `chown` is done - Warnings collect in `OrganizationResult.permissionWarnings` / `EbookOrganizationResult.permissionWarnings` (never in `errors`); `reportImportStatus` logs one summary per job (`summarizePermissionWarnings`) and the request message gets `permissions could not be applied to <n> file(s)` -**Ebook placement record (#497):** after a successful ebook organize, `recordEbookPlacement` (`src/lib/utils/ebook-placement.ts`, one call in `processEbookOrganization`) appends `{ path, size, sha256, placedAt }` of the placed target to `Request.placedEbookFiles`, only when placeFile returned `copied` (RMAB created the file; hash read from the placed bytes, after the Kindle fix). `identical`/`present`/`differs` add nothing (field becomes at least `[]`). Append-only, same path replaced by the newest entry; never fails the import (a hash or DB error is a warning and leaves the field as it was). The ebook request delete removes only these files ([request-deletion.md](../admin-features/request-deletion.md) D2). +**Ebook placement record (#497):** after a successful ebook organize, `recordEbookPlacement` (`src/lib/utils/ebook-placement.ts`, one call in `processEbookOrganization`) appends `{ path, size, sha256, placedAt }` of the placed target to `Request.placedEbookFiles`, only when placeFile returned `copied` (RMAB created the file; hash read from the placed bytes, after the Kindle fix). `identical`/`present`/`differs` add nothing (field becomes at least `[]`). Append-only, same path replaced by the newest entry; a NULL record on a request whose `completedAt` is already set (an earlier delivery with no record) starts with `{ earlierImport: 'unrecorded' }`; never fails the import (a hash or DB error is a warning and leaves the field as it was). The ebook request delete removes only these files ([request-deletion.md](../admin-features/request-deletion.md) D2). **Stale partials:** before placing, regular files in that one target folder matching the exact pattern are removed when they belong to this target name or are older than 24 h. diff --git a/documentation/testing/e2e-harness.md b/documentation/testing/e2e-harness.md index 10504f885..bb3c43f4f 100644 --- a/documentation/testing/e2e-harness.md +++ b/documentation/testing/e2e-harness.md @@ -120,8 +120,8 @@ | `regressions/summary-scroll.spec.ts` | Book summary box: details-modal scroll-area overflow (`scrollHeight - clientHeight`) for 1.8k/3.9k/8.1k-char real Audnexus summaries (`summary-scroll-fixtures.json`, stripHtml output) at 375x812, 812x375 (touch), 1366x657, 1440x900, 2560x1440, plus 2:3 covers and the admin approval modal; edge cases (one paragraph, box-set list, 80-char unbroken word, 9/10 lines at the cap, whitespace only); wheel and CDP touch swipe at the box end move the modal; "more" chevron: disc top meets the fade top, never clipped or past the next divider, each click scrolls the box (not the modal) by its height − 40 until it goes; "-end" shots wait two frames and check the chevron and fade are gone; focused region keeps focus when it stops overflowing (Chromium); helpers in `summary-scroll-helpers.ts` (setup, geometry) and `summary-scroll-input.ts` (real input); Tab + PageDown; axe `scrollable-region-focusable` (axe from the pinned `axe-core` e2e devDependency, injected by `page.evaluate`); short/long at 1440x900 and 375x812 repeated in WebKit and Firefox (launched in-spec from the Playwright image). Details responses are the live ones with `description` swapped via `page.route`; search responses replayed. `metrics.json` + PNGs under `summary-scroll/`; admin run needs the exec bridge | | `regressions/423-takeover.spec.ts` | #423: B grabbed normally, A and C already in qBittorrent (`qbit-seed`, 2 days before the grab); indexer seeds 1 min. A/C rows `added_by_rmab=false`, B `true`; admin delete of C keeps it (`torrentsKeptNotOwned: 1`, no `torrents/delete`); seeding cleanup removes only B (row → `removed`), A/C files keep the fixture sha256, C's soft-deleted request is hard-deleted without a client call; exec bridge required. Uses all three fixture books | | `regressions/423-upgrade-takeover.spec.ts` | #423 across an upgrade (`--upgrade-from 1.2.10 --upgrade-specs 423-upgrade`): on the release image B normal + A taken over, seeding unlimited so the old cleanup (triggered once) keeps both; after the upgrade the column exists, both rows NULL; with a 1-min rule the cleanup removes B and keeps A (recorded `false`, sha256 = fixture). Not named `*upgrade.spec.ts`: the stock run's filter `upgrade.spec.ts` is a path regex and would pick it up | -| `regressions/497-ebook-delete.spec.ts` | #497, an ebook request delete removes only the ebook file that request placed (audio by manual import, ebooks by the app's organize job): (a) the user's epub in an imported title folder + a `failed` ebook request (SQL) → nothing removed, no reason; (b) a real import of `Book.epub` beside the user's `Book.azw3` → `placed_ebook_files` holds the epub (path, sha256), delete removes only it; (c) the user's `Book.epub`, then a real import of a same-name, same-size, different-bytes epub (`present`) → record `[]`, request `downloaded`, delete keeps the user's file byte for byte with the "already in the library" reason. Evidence under `library-delete/497-*`; helpers in `497-helpers.ts`; exec bridge required | -| `regressions/497-ebook-legacy.spec.ts` | #497 across an upgrade (`--upgrade-from 1.2.11 --upgrade-specs 497-ebook-legacy`): on the release image L1 a delivered ebook import, L2 the user's epub + a `failed` ebook request, L3 a `pending` ebook request; after the upgrade `placed_ebook_files` exists and is NULL on all three; L1's delete removes nothing (reason: imported before RMAB kept records), L2's removes nothing, no reason; L3 organized into its existing row (`organizeEbookInto`: no new `requests`/`audiobooks` row) records its epub and its delete removes only that. Not named `*upgrade.spec.ts` (see above) | +| `regressions/497-ebook-delete.spec.ts` | #497, an ebook request delete removes only the ebook file that request placed (audio by manual import, ebooks by the app's organize job): (a) the user's epub in an imported title folder + a `failed` ebook request (SQL) → nothing removed, no reason; (b) a real import of `Book.epub` beside the user's `Book.azw3` → `placed_ebook_files` holds the epub (path, sha256), delete removes only it; (c) the user's `Book.epub`, then a real import of a same-name, same-size, different-bytes epub (`present`) → record `[]`, request `downloaded`, delete keeps the user's file byte for byte with the "its import found it already in the library" reason. Evidence under `library-delete/497-*`; helpers in `497-helpers.ts`; exec bridge required | +| `regressions/497-ebook-legacy.spec.ts` | #497 across an upgrade (`--upgrade-from 1.2.12 --upgrade-specs 497-ebook-legacy`): on the release image L1 a delivered ebook import, L2 the user's epub + a `failed` ebook request, L3 a `pending` ebook request; after the upgrade `placed_ebook_files` exists and is NULL on all three; L1's delete removes nothing (reason: RMAB has no record of the ebook files this request placed), L2's removes nothing, no reason; L3 organized into its existing row (`organizeEbookInto`: no new `requests`/`audiobooks` row) records its epub and its delete removes only that. Not named `*upgrade.spec.ts` (see above) | | `regressions/434-multi-disc-import.spec.ts` | #434 (`--offline-fixtures --overrides abs-fake-metadata`): A Manual Import UI, tagging on, merge off: a disc-only folder with mixed and chapter-title names (footer "in 2 discs"), 7 flat files (`Disc 1-01.m4a`, `Kingdoms Fall.m4a`…), `Extras/` stays out, each file tagged disc `d/2` + track `n/7`, Audiobookshelf `audioFiles` by `index` in disc→track order; A2 control, tagging off: the same names alone are not in that order; B merge on, API without `selectedFiles`: chapter durations run Intro, Disc 1, Disc 2; C Bulk scan + execute: untagged `CD1/`+`CD2/` = one entry, one success; D single-folder names + `filesHash` unchanged, traversal/`Extras/` entries 400. B-D tagging off. Values read first + `expect.soft`; helpers in `434-helpers.ts`; exec bridge required | | `regressions/278-ebook-manual-import.spec.ts` | #278 (`--offline-fixtures --overrides abs-fake-metadata`): one fake-ASIN book, its audiobook imported (generated mp3) and set `available`, plus a failed ebook request; a download folder with a generated mp3 and an epub stand-in. UI: My Requests ebook card, details modal, Manual Import, cleanup on: the browser lists only the epub ("1 ebook file"), sends `requestId` + `requestType: ebook`; the epub lands in the book folder (same bytes), the ebook request is `downloaded`, the audiobook request row, the book row path/files hash and the library mp3 are unchanged, no organize job for the audiobook request, the source epub removed and the mp3 and folder kept. Values read first + `expect.soft` (`278/result.json`); exec bridge required | | `regressions/302-watch-new-releases.spec.ts` | #302 UI and persistence (the release filter itself is unit-tested): made-up author (detail + books via `page.route`); Watch dialog preselects and focuses Entire catalog, ArrowDown + Space picks New releases only, POST sends `newReleasesOnly: true`, `new_releases_since` set; a series watched over the API reads NULL; Profile cards show "New releases since …" / "Entire catalog"; Change → Save disabled until changed, Entire catalog shows the backlog note, Save PATCHes and clears the cutoff (API + psql); snapshots at 1280 px, 375 px, dark and 375 dark; exec bridge required | diff --git a/e2e/specs/regressions/497-ebook-legacy.spec.ts b/e2e/specs/regressions/497-ebook-legacy.spec.ts index f66416bc6..121e72d74 100644 --- a/e2e/specs/regressions/497-ebook-legacy.spec.ts +++ b/e2e/specs/regressions/497-ebook-legacy.spec.ts @@ -2,7 +2,7 @@ * Component: E2E regression #497 across an upgrade (ebook requests imported before the placement record) * Documentation: documentation/testing/e2e-harness.md * - * run.mjs --upgrade-from 1.2.11 --upgrade-specs 497-ebook-legacy runs this file on the release image + * run.mjs --upgrade-from 1.2.12 --upgrade-specs 497-ebook-legacy runs this file on the release image * (E2E_PHASE=upgrade-seed), then on the branch image on the same volumes (upgrade-verify). * seed: L1 an audiobook + a delivered ebook import (the release copies the epub, keeps no record); * L2 an audiobook + the user's own epub in its folder + a failed ebook request (SQL); diff --git a/e2e/specs/regressions/497-helpers.ts b/e2e/specs/regressions/497-helpers.ts index af2a6149b..b8c9d2239 100644 --- a/e2e/specs/regressions/497-helpers.ts +++ b/e2e/specs/regressions/497-helpers.ts @@ -5,8 +5,8 @@ import { asApp, psql, q, sh, sqlStr } from './library-delete-helpers'; -export const LEGACY_REASON = 'this request was imported before RMAB kept a record of the ebook files it places, so no ebook file was removed'; -export const ALREADY_THERE_REASON = 'the ebook was already in the library when this request imported it, so it was not removed'; +export const LEGACY_REASON = 'RMAB has no record of the ebook files this request placed (it was imported before RMAB kept one, or the record could not be saved), so no ebook file was removed'; +export const ALREADY_THERE_REASON = 'RMAB has no record that this request placed the ebook (its import found it already in the library), so it was not removed'; export function hasPlacedColumn(): boolean { return psql(`select count(*) from information_schema.columns where table_name = 'requests' and column_name = 'placed_ebook_files'`) === '1'; diff --git a/src/app/api/admin/reported-issues/[id]/replace/route.ts b/src/app/api/admin/reported-issues/[id]/replace/route.ts index cb3923e0a..f06e1c7c7 100644 --- a/src/app/api/admin/reported-issues/[id]/replace/route.ts +++ b/src/app/api/admin/reported-issues/[id]/replace/route.ts @@ -11,6 +11,7 @@ import { z } from 'zod'; import { errorForLog, zodIssueDetails } from '@/lib/utils/zod-validation'; import { RMABLogger } from '@/lib/utils/logger'; import { readJsonBody } from '@/lib/utils/request-body'; +import { hidePlacementRecord } from '@/lib/services/request-detail-query'; const logger = RMABLogger.create('API.Admin.ReportedIssues.Replace'); @@ -65,7 +66,7 @@ export async function POST( return NextResponse.json({ success: true, - request: result.request, + request: hidePlacementRecord(result.request), }); } catch (error) { if (error instanceof z.ZodError) { diff --git a/src/app/api/admin/requests/[id]/approve/route.ts b/src/app/api/admin/requests/[id]/approve/route.ts index 6fac4e551..a2bf98a95 100644 --- a/src/app/api/admin/requests/[id]/approve/route.ts +++ b/src/app/api/admin/requests/[id]/approve/route.ts @@ -14,6 +14,7 @@ import { buildEbookDownloadName } from '@/lib/utils/ebook-download-name'; import { z } from 'zod'; import { errorForLog, zodIssueDetails } from '@/lib/utils/zod-validation'; import { readJsonBody } from '@/lib/utils/request-body'; +import { hidePlacementRecord } from '@/lib/services/request-detail-query'; const logger = RMABLogger.create('API.Admin.Requests.Approve'); @@ -219,7 +220,7 @@ export async function POST( message: adminSelectedTorrent ? 'Request approved and download started with admin-selected torrent' : 'Request approved and download started with pre-selected torrent', - request: updatedRequest, + request: hidePlacementRecord(updatedRequest), }); } else { // No pre-selected torrent - use automatic search @@ -293,7 +294,7 @@ export async function POST( ? `The user's selected release could not be used (${droppedPickReason}). ${searchMessage} instead.` : searchMessage, ...(droppedPickReason ? { selectionDropped: droppedPickReason } : {}), - request: updatedRequest, + request: hidePlacementRecord(updatedRequest), }); } } else { @@ -322,7 +323,7 @@ export async function POST( return NextResponse.json({ success: true, message: 'Request denied', - request: updatedRequest, + request: hidePlacementRecord(updatedRequest), }); } } catch (error) { diff --git a/src/app/api/admin/requests/pending-approval/route.ts b/src/app/api/admin/requests/pending-approval/route.ts index 00161e56c..7742aaf4b 100644 --- a/src/app/api/admin/requests/pending-approval/route.ts +++ b/src/app/api/admin/requests/pending-approval/route.ts @@ -7,6 +7,7 @@ import { NextRequest, NextResponse } from 'next/server'; import { requireAuth, requireAdmin, AuthenticatedRequest } from '@/lib/middleware/auth'; import { prisma } from '@/lib/db'; import { RMABLogger } from '@/lib/utils/logger'; +import { hidePlacementRecord } from '@/lib/services/request-detail-query'; const logger = RMABLogger.create('API.Admin.Requests.PendingApproval'); @@ -38,7 +39,7 @@ export async function GET(request: NextRequest) { return NextResponse.json({ success: true, - requests, + requests: requests.map((r) => hidePlacementRecord(r)), count: requests.length, }); } catch (error) { diff --git a/src/app/api/requests/[id]/manual-search/route.ts b/src/app/api/requests/[id]/manual-search/route.ts index c93fb0682..c767a8e62 100644 --- a/src/app/api/requests/[id]/manual-search/route.ts +++ b/src/app/api/requests/[id]/manual-search/route.ts @@ -8,6 +8,7 @@ import { requireAuth, AuthenticatedRequest } from '@/lib/middleware/auth'; import { prisma } from '@/lib/db'; import { getJobQueueService } from '@/lib/services/job-queue.service'; import { RMABLogger } from '@/lib/utils/logger'; +import { hidePlacementRecord } from '@/lib/services/request-detail-query'; const logger = RMABLogger.create('API.ManualSearch'); @@ -97,7 +98,7 @@ export async function POST( return NextResponse.json({ success: true, - request: updated, + request: hidePlacementRecord(updated), message: 'Manual search initiated', }); } catch (error) { diff --git a/src/app/api/requests/[id]/route.ts b/src/app/api/requests/[id]/route.ts index d7105c498..a30e9dcd7 100644 --- a/src/app/api/requests/[id]/route.ts +++ b/src/app/api/requests/[id]/route.ts @@ -10,7 +10,7 @@ import { prisma } from '@/lib/db'; import { RMABLogger } from '@/lib/utils/logger'; import { CLIENT_PROTOCOL_MAP, DownloadClientType } from '@/lib/interfaces/download-client.interface'; import { CANCELLABLE_STATUSES } from '@/lib/constants/request-statuses'; -import { ADMIN_REQUEST_DETAIL_INCLUDE, OWNER_REQUEST_DETAIL_SELECT } from '@/lib/services/request-detail-query'; +import { ADMIN_REQUEST_DETAIL_INCLUDE, OWNER_REQUEST_DETAIL_SELECT, hidePlacementRecord } from '@/lib/services/request-detail-query'; import { stripProwlarrKeyFromResults } from '@/lib/services/download-selection-guard'; import { readJsonBody } from '@/lib/utils/request-body'; import { toJsonSafe } from '@/lib/utils/json-safe'; @@ -62,7 +62,7 @@ export async function GET( // Sizes are BigInt columns, which JSON.stringify cannot serialize (#261) return NextResponse.json({ success: true, - request: toJsonSafe(isAdmin ? requestRecord : await stripProwlarrKeyFromResults(requestRecord)), + request: toJsonSafe(isAdmin ? hidePlacementRecord(requestRecord) : await stripProwlarrKeyFromResults(requestRecord)), }); } catch (error) { logger.error('Failed to get request', { error: error instanceof Error ? error.message : String(error) }); @@ -167,7 +167,7 @@ export async function PATCH( return NextResponse.json({ success: true, - request: updated, + request: hidePlacementRecord(updated), message: 'Request cancelled successfully', }); } else if (action === 'retry') { @@ -335,7 +335,7 @@ export async function PATCH( return NextResponse.json({ success: true, - request: updated, + request: hidePlacementRecord(updated), message: `Request retry initiated (${jobType})`, }); } diff --git a/src/app/api/requests/route.ts b/src/app/api/requests/route.ts index 1d982bf6a..76e94e91a 100644 --- a/src/app/api/requests/route.ts +++ b/src/app/api/requests/route.ts @@ -12,6 +12,7 @@ import { RMABLogger } from '@/lib/utils/logger'; import { createRequestForUser } from '@/lib/services/request-creator.service'; import { COMPLETED_STATUSES } from '@/lib/constants/request-statuses'; import { readJsonBody } from '@/lib/utils/request-body'; +import { hidePlacementRecord } from '@/lib/services/request-detail-query'; const logger = RMABLogger.create('API.Requests'); @@ -76,7 +77,7 @@ export async function POST(request: NextRequest) { return NextResponse.json({ success: true, - request: result.request, + request: hidePlacementRecord(result.request), }, { status: 201 }); } catch (error) { logger.error('Failed to create request', { error: errorForLog(error) }); @@ -192,7 +193,8 @@ export async function GET(request: NextRequest) { const audiobook = r.audiobook ? { ...r.audiobook, filePath: undefined } : r.audiobook; // Stored release picks are internal (they can carry source credentials); admins only const selectedTorrent = isAdmin ? r.selectedTorrent : undefined; - return { ...r, selectedTorrent, audiobook, downloadAvailable }; + // The ebook placement record (absolute library paths, #497) is for the request delete only + return { ...r, selectedTorrent, audiobook, downloadAvailable, placedEbookFiles: undefined }; }); // ── Accurate counts per group (always scoped to ownership/type filter) ── diff --git a/src/lib/services/request-delete-files.ts b/src/lib/services/request-delete-files.ts index ad6720748..b28efe4e0 100644 --- a/src/lib/services/request-delete-files.ts +++ b/src/lib/services/request-delete-files.ts @@ -15,7 +15,7 @@ import { prisma } from '../db'; import { RMABLogger } from '../utils/logger'; import { canonicalPath, UnsafeDeleteError } from '../utils/delete-guard'; import { assertSafeLibraryDelete, isStaleStoredFolder, type LibraryBook } from '../utils/library-delete-guard'; -import { otherRecordedPaths, planPlacedEbookDelete, readPlacements } from '../utils/ebook-placement'; +import { otherRecordedEntries, planPlacedEbookDelete, readPlacements } from '../utils/ebook-placement'; import { sweepEmptyParents } from '../processors/organize-cleanup-guard'; import { isPathBelowBoundary } from '../utils/cleanup-helpers'; import { libraryRelativePath, normalizeRelPath } from '../utils/library-item-path'; @@ -23,8 +23,10 @@ import { libraryRelativePath, normalizeRelPath } from '../utils/library-item-pat const logger = RMABLogger.create('RequestDelete'); const NO_FOLDER_REASON = 'RMAB has no stored folder for this book, so its library files were not removed'; -const LEGACY_EBOOK_REASON = 'this request was imported before RMAB kept a record of the ebook files it places, so no ebook file was removed'; -const EBOOK_ALREADY_THERE_REASON = 'the ebook was already in the library when this request imported it, so it was not removed'; +const NO_RECORD_WHY = 'it was imported before RMAB kept one, or the record could not be saved'; +const LEGACY_EBOOK_REASON = `RMAB has no record of the ebook files this request placed (${NO_RECORD_WHY}), so no ebook file was removed`; +const EARLIER_IMPORT_REASON = `an earlier import of this request has no record (${NO_RECORD_WHY}), so its ebook file was not removed`; +const EBOOK_ALREADY_THERE_REASON = 'RMAB has no record that this request placed the ebook (its import found it already in the library), so it was not removed'; const UNREADABLE_RECORD_REASON = 'the placement record could not be read'; export interface DeleteBook extends LibraryBook { @@ -114,16 +116,18 @@ export async function removeEbookFiles(book: DeleteBook, request: EbookDeleteReq logger.info(`No ebook placement record for request ${request.id} ("${book.title}"); no ebook file removed`); return delivered ? { deleted: 0, keptReason: LEGACY_EBOOK_REASON } : { deleted: 0 }; } - if (record.entries.length === 0 && record.unreadable === 0) { + if (record.entries.length === 0 && record.unreadable === 0 && !record.unrecordedEarlier) { logger.info(`Request ${request.id} placed no ebook file ("${book.title}": the library already had it); none removed`); return { deleted: 0, keptReason: EBOOK_ALREADY_THERE_REASON }; } - const reasons: string[] = record.unreadable > 0 ? [UNREADABLE_RECORD_REASON] : []; - const otherPaths = record.entries.length > 0 ? await otherRecordedPaths(request.id) : new Set<string>(); + const reasons: string[] = []; + if (record.unrecordedEarlier) reasons.push(EARLIER_IMPORT_REASON); + if (record.unreadable > 0) reasons.push(UNREADABLE_RECORD_REASON); + const others = record.entries.length > 0 ? await otherRecordedEntries(request.id) : []; let deleted = 0; for (const entry of record.entries) { - const decision = await planPlacedEbookDelete(entry, book, otherPaths); + const decision = await planPlacedEbookDelete(entry, book, others); if (decision.kind === 'kept') { reasons.push(decision.reason); } else if (decision.kind === 'gone') { @@ -141,7 +145,8 @@ export async function removeEbookFiles(book: DeleteBook, request: EbookDeleteReq } } if (reasons.length === 0) return { deleted }; - const keptReason = reasons.join('; '); + // Several entries can share one reason (e.g. two files in a folder that fails rule c): each shows once + const keptReason = [...new Set(reasons)].join('; '); logger.warn(`Keeping ebook file(s) of request ${request.id} ("${book.title}"): ${keptReason}`); return { deleted, keptReason }; } diff --git a/src/lib/services/request-detail-query.ts b/src/lib/services/request-detail-query.ts index 0d0cdeb25..832ddbed5 100644 --- a/src/lib/services/request-detail-query.ts +++ b/src/lib/services/request-detail-query.ts @@ -5,7 +5,7 @@ * Non-admin callers (owners, API tokens) get an explicit field list that leaves out * internal job/download data: Job.payload/result/stackTrace, * DownloadHistory.magnetLink/torrentUrl/downloadPath, Audiobook.filePath and - * Request.selectedTorrent. + * Request.selectedTorrent. Request.placedEbookFiles is left out for every caller. */ import type { Prisma } from '@/generated/prisma/client'; @@ -118,6 +118,17 @@ export const OWNER_REQUEST_DETAIL_SELECT = { * internal (download link) and only admins act on it. */ export function hideStoredPick<T>(record: T, role: string | undefined): T { - if (role === 'admin' || !record || typeof record !== 'object') return record; - return { ...record, selectedTorrent: undefined }; + const shown = hidePlacementRecord(record); + if (role === 'admin' || !shown || typeof shown !== 'object') return shown; + return { ...shown, selectedTorrent: undefined }; +} + +/** + * Drop the ebook placement record (Request.placedEbookFiles, #497: absolute + * library paths and hashes) from a request returned to any caller, admins + * included. Only the request delete reads it. + */ +export function hidePlacementRecord<T>(record: T): T { + if (!record || typeof record !== 'object') return record; + return { ...record, placedEbookFiles: undefined }; } diff --git a/src/lib/utils/ebook-placement.ts b/src/lib/utils/ebook-placement.ts index fdd0c4005..49bd63e18 100644 --- a/src/lib/utils/ebook-placement.ts +++ b/src/lib/utils/ebook-placement.ts @@ -7,13 +7,14 @@ * placeFile copied the file ('copied'); 'identical', 'present' and 'differs' * mean the library already had it, so the import adds nothing (the field is then * at least []). NULL means no record (never imported, or imported before this - * field existed). A request delete removes only recorded files, and only while - * each is still the exact file RMAB placed (size + sha256) and its folder passes - * library delete guard rules a-c. + * field existed). A record whose request had already delivered before its first + * recorded import starts with UNRECORDED_EARLIER_IMPORT. A request delete removes + * only recorded files, and only while each is still the exact file RMAB placed + * (size + sha256) and its folder passes library delete guard rules a-c. */ import { createHash } from 'crypto'; -import { createReadStream } from 'fs'; +import { createReadStream, type BigIntStats } from 'fs'; import * as fs from 'fs/promises'; import * as path from 'path'; import { prisma } from '../db'; @@ -36,10 +37,19 @@ export interface PlacementRecordRead { entries: EbookPlacement[]; /** Items that are not a well-formed entry: their file is never removed. */ unreadable: number; + /** An earlier import of this request delivered with no record (before the field, or a failed write). */ + unrecordedEarlier: boolean; } +/** Marker for "this request delivered before its first recorded import": never a file to remove. */ +export const UNRECORDED_EARLIER_IMPORT = { earlierImport: 'unrecorded' } as const; + const SHA256 = /^[0-9a-f]{64}$/; +function isUnrecordedMarker(value: unknown): boolean { + return !!value && typeof value === 'object' && (value as { earlierImport?: unknown }).earlierImport === 'unrecorded'; +} + function isPlacement(value: unknown): value is EbookPlacement { if (!value || typeof value !== 'object' || Array.isArray(value)) return false; const v = value as Record<string, unknown>; @@ -58,9 +68,10 @@ function isPlacement(value: unknown): value is EbookPlacement { /** The record as stored; null when there is none (SQL NULL and JSON null alike). */ export function readPlacements(value: unknown): PlacementRecordRead | null { if (value === null || value === undefined) return null; - if (!Array.isArray(value)) return { entries: [], unreadable: 1 }; + if (!Array.isArray(value)) return { entries: [], unreadable: 1, unrecordedEarlier: false }; + const markers = value.filter(isUnrecordedMarker).length; const entries = value.filter(isPlacement); - return { entries, unreadable: value.length - entries.length }; + return { entries, unreadable: value.length - entries.length - markers, unrecordedEarlier: markers > 0 }; } async function sha256File(file: string): Promise<string> { @@ -76,9 +87,14 @@ export async function placementEntry(file: string): Promise<EbookPlacement> { return { path: path.resolve(file), size: stat.size, sha256: await sha256File(file), placedAt: new Date().toISOString() }; } -/** Append `entry` to the stored value (same path: newest wins); anything stored is kept. */ -export function mergePlacements(stored: unknown, entry: EbookPlacement | null): unknown[] { - const existing = Array.isArray(stored) ? stored : stored === null || stored === undefined ? [] : [stored]; +/** + * Append `entry` to the stored value (same path: newest wins); anything stored is kept. + * No stored record but `unrecordedEarlier` (the request delivered before): the record + * starts with UNRECORDED_EARLIER_IMPORT. + */ +export function mergePlacements(stored: unknown, entry: EbookPlacement | null, unrecordedEarlier = false): unknown[] { + const none = stored === null || stored === undefined; + const existing = Array.isArray(stored) ? stored : none ? (unrecordedEarlier ? [{ ...UNRECORDED_EARLIER_IMPORT }] : []) : [stored]; if (!entry) return existing; const samePath = (item: unknown) => !!item && typeof item === 'object' && (item as { path?: unknown }).path === entry.path; return [...existing.filter((item) => !samePath(item)), entry]; @@ -92,10 +108,12 @@ export function mergePlacements(stored: unknown, entry: EbookPlacement | null): export async function recordEbookPlacement(requestId: string, record: PlacementRecord | undefined, logger: RMABLogger): Promise<void> { try { const entry = record?.status === 'copied' ? await placementEntry(record.target) : null; - const row = await prisma.request.findUnique({ where: { id: requestId }, select: { placedEbookFiles: true } }); + const row = await prisma.request.findUnique({ where: { id: requestId }, select: { placedEbookFiles: true, completedAt: true } }); + // completedAt is written after this record, so a value here is an earlier delivery + const unrecordedEarlier = (row?.placedEbookFiles === null || row?.placedEbookFiles === undefined) && !!row?.completedAt; await prisma.request.update({ where: { id: requestId }, - data: { placedEbookFiles: mergePlacements(row?.placedEbookFiles, entry) as object[] }, + data: { placedEbookFiles: mergePlacements(row?.placedEbookFiles, entry, unrecordedEarlier) as object[] }, }); logger.info( entry @@ -110,20 +128,35 @@ export async function recordEbookPlacement(requestId: string, record: PlacementR } /** - * Library paths recorded by every other active request, of any book. A file two + * Entries recorded by every other active request, of any book. A file two * requests recorded stays until neither needs it. Only the ebook organize writes * records, so only ebook requests can hold one. */ -export async function otherRecordedPaths(requestId: string): Promise<Set<string>> { +export async function otherRecordedEntries(requestId: string): Promise<EbookPlacement[]> { const rows = await prisma.request.findMany({ where: { id: { not: requestId }, deletedAt: null, type: 'ebook' }, select: { placedEbookFiles: true }, }); - const paths = new Set<string>(); - for (const row of rows) { - for (const entry of readPlacements(row.placedEbookFiles)?.entries ?? []) paths.add(path.resolve(entry.path)); + return rows.flatMap((row) => readPlacements(row.placedEbookFiles)?.entries ?? []); +} + +/** + * Whether another request's entry is this very file: same sha256 and size, and its + * path reaches the same file (dev/ino), however that path is spelled (symlinked + * folder, another case on a case-insensitive mount). An error code when one could not be checked. + */ +async function recordedByAnother(file: BigIntStats, entry: EbookPlacement, others: EbookPlacement[]): Promise<boolean | string> { + for (const other of others) { + if (other.sha256 !== entry.sha256 || other.size !== entry.size) continue; + try { + const s = await fs.lstat(other.path, { bigint: true }); + if (s.isFile() && s.dev === file.dev && s.ino === file.ino) return true; + } catch (error) { + const code = (error as NodeJS.ErrnoException).code ?? 'error'; + if (code !== 'ENOENT' && code !== 'ENOTDIR') return code; + } } - return paths; + return false; } export type PlacedFileDecision = @@ -137,11 +170,9 @@ export type PlacedFileDecision = export async function planPlacedEbookDelete( entry: EbookPlacement, book: LibraryBook, - otherPaths: Set<string> + others: EbookPlacement[] ): Promise<PlacedFileDecision> { const name = path.basename(entry.path); - if (otherPaths.has(path.resolve(entry.path))) return { kind: 'kept', reason: `${name} is also the ebook of another request` }; - let safe; try { safe = await assertSafeLibraryDelete(path.dirname(entry.path), book, { evidence: false }); @@ -152,16 +183,20 @@ export async function planPlacedEbookDelete( if (!safe.exists) return { kind: 'gone' }; const file = path.join(safe.canonicalPath, name); + let shared: boolean | string; try { - const stat = await fs.lstat(file); + const stat = await fs.lstat(file, { bigint: true }); if (!stat.isFile()) return { kind: 'kept', reason: `${name} is no longer the file RMAB placed` }; - if (stat.size !== entry.size || (await sha256File(file)) !== entry.sha256) { + if (Number(stat.size) !== entry.size || (await sha256File(file)) !== entry.sha256) { return { kind: 'kept', reason: `${name} changed since RMAB placed it` }; } + shared = await recordedByAnother(stat, entry, others); } catch (error) { const code = (error as NodeJS.ErrnoException).code ?? 'error'; if (code === 'ENOENT' || code === 'ENOTDIR') return { kind: 'gone' }; return { kind: 'kept', reason: `${name} could not be checked (${code})` }; } + if (shared === true) return { kind: 'kept', reason: `${name} is also the ebook of another request` }; + if (shared) return { kind: 'kept', reason: `${name} could not be compared with another request's record (${shared})` }; return { kind: 'remove', file }; } diff --git a/tests/api/requests-placement-record.routes.test.ts b/tests/api/requests-placement-record.routes.test.ts new file mode 100644 index 000000000..6873110db --- /dev/null +++ b/tests/api/requests-placement-record.routes.test.ts @@ -0,0 +1,143 @@ +/** + * Component: Request responses leave out the ebook placement record (#497) + * Documentation: documentation/admin-features/request-deletion.md + * + * Request.placedEbookFiles holds absolute library paths and hashes for the request + * delete only. No response that returns a request row carries it, for users, + * API tokens and admins alike. + */ + +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import { createPrismaMock } from '../helpers/prisma'; + +let authRequest: { user: { id: string; role: string }; nextUrl: URL; json: ReturnType<typeof vi.fn> }; + +const prismaMock = createPrismaMock(); +const requireAuthMock = vi.hoisted(() => vi.fn()); +const requireAdminMock = vi.hoisted(() => vi.fn()); +const createRequestForUserMock = vi.hoisted(() => vi.fn()); +const jobQueueMock = vi.hoisted(() => ({ addSearchJob: vi.fn(), addNotificationJob: vi.fn().mockResolvedValue(undefined) })); + +vi.mock('@/lib/db', () => ({ prisma: prismaMock })); +vi.mock('@/lib/middleware/auth', () => ({ requireAuth: requireAuthMock, requireAdmin: requireAdminMock })); +vi.mock('@/lib/services/request-creator.service', () => ({ createRequestForUser: createRequestForUserMock })); +vi.mock('@/lib/services/job-queue.service', () => ({ getJobQueueService: () => jobQueueMock })); +vi.mock('@/lib/services/config.service', () => ({ getConfigService: () => ({}) })); +vi.mock('@/lib/integrations/prowlarr-connection', () => ({ + getProwlarrConnection: async () => ({ + baseUrl: 'http://prowlarr:9696', + apiKey: 'test-prowlarr-key', + base: { origin: 'http://prowlarr:9696', protocol: 'http:', hostname: 'prowlarr', host: 'prowlarr:9696', port: '9696', defaultPort: false, userinfo: '', basePath: '' }, + }), +})); + +const { hidePlacementRecord, hideStoredPick } = await import('@/lib/services/request-detail-query'); + +const PLACED = [{ path: '/media/Author/Book/Book.epub', size: 10, sha256: 'a'.repeat(64), placedAt: '2026-10-01T00:00:00.000Z' }]; + +/** A delivered ebook request row as Prisma returns it (include, no select). */ +function row(overrides: Record<string, unknown> = {}) { + return { + id: 'req-1', + userId: 'user-1', + type: 'ebook', + status: 'downloaded', + selectedTorrent: null, + placedEbookFiles: PLACED, + audiobook: { id: 'ab-1', title: 'Book', author: 'Author', filePath: '/media/Author/Book' }, + user: { id: 'user-1', plexUsername: 'user' }, + ...overrides, + }; +} + +function as(role: 'user' | 'admin', url = 'http://localhost/api/requests') { + authRequest = { user: { id: role === 'admin' ? 'admin-1' : 'user-1', role }, nextUrl: new URL(url), json: vi.fn() }; +} + +beforeEach(() => { + vi.clearAllMocks(); + as('user'); + requireAuthMock.mockImplementation((_req: unknown, handler: (req: unknown) => unknown) => handler(authRequest)); + requireAdminMock.mockImplementation((_req: unknown, handler: () => unknown) => handler()); + prismaMock.request.count.mockResolvedValue(1); +}); + +describe('GET /api/requests (on the API-token allowlist)', () => { + it.each(['user', 'admin'] as const)('a %s gets no placedEbookFiles (and no audiobook.filePath)', async (role) => { + as(role, 'http://localhost/api/requests?type=ebook'); + prismaMock.request.findMany.mockResolvedValueOnce([row()]); + + const { GET } = await import('@/app/api/requests/route'); + const payload = await (await GET({} as never)).json(); + + expect(payload.requests).toHaveLength(1); + expect(payload.requests[0].id).toBe('req-1'); + expect('placedEbookFiles' in payload.requests[0]).toBe(false); + expect('filePath' in payload.requests[0].audiobook).toBe(false); + expect(JSON.stringify(payload)).not.toContain('Book.epub'); + }); +}); + +describe('other responses that return a request row', () => { + it('POST /api/requests', async () => { + authRequest.json.mockResolvedValue({ audiobook: { asin: 'B0TEST0001', title: 'Book', author: 'Author' } }); + createRequestForUserMock.mockResolvedValueOnce({ success: true, request: row({ type: 'audiobook', status: 'pending', placedEbookFiles: null }) }); + + const { POST } = await import('@/app/api/requests/route'); + const response = await POST({} as never); + const payload = await response.json(); + + expect(response.status).toBe(201); + expect(payload.request.id).toBe('req-1'); + expect('placedEbookFiles' in payload.request).toBe(false); + }); + + it('GET /api/requests/[id] for an admin (full row)', async () => { + as('admin', 'http://localhost/api/requests/req-1'); + prismaMock.request.findFirst.mockResolvedValueOnce(row({ downloadHistory: [], jobs: [] })); + + const { GET } = await import('@/app/api/requests/[id]/route'); + const payload = await (await GET({} as never, { params: Promise.resolve({ id: 'req-1' }) })).json(); + + expect(payload.request.id).toBe('req-1'); + expect('placedEbookFiles' in payload.request).toBe(false); + }); + + it('PATCH /api/requests/[id] (cancel)', async () => { + authRequest.json.mockResolvedValue({ action: 'cancel' }); + prismaMock.request.findFirst.mockResolvedValueOnce(row({ status: 'pending' })); + prismaMock.request.update.mockResolvedValueOnce(row({ status: 'cancelled' })); + + const { PATCH } = await import('@/app/api/requests/[id]/route'); + const payload = await (await PATCH({} as never, { params: Promise.resolve({ id: 'req-1' }) })).json(); + + expect(payload.request.status).toBe('cancelled'); + expect('placedEbookFiles' in payload.request).toBe(false); + }); + + it('GET /api/admin/requests/pending-approval', async () => { + as('admin', 'http://localhost/api/admin/requests/pending-approval'); + prismaMock.request.findMany.mockResolvedValueOnce([row({ status: 'awaiting_approval' })]); + + const { GET } = await import('@/app/api/admin/requests/pending-approval/route'); + const payload = await (await GET({} as never)).json(); + + expect(payload.requests[0].id).toBe('req-1'); + expect('placedEbookFiles' in payload.requests[0]).toBe(false); + }); +}); + +describe('hidePlacementRecord / hideStoredPick (approve, manual-search, select-torrent, request-with-torrent, reported-issue replace)', () => { + it('hidePlacementRecord drops only the record', () => { + const shown = JSON.parse(JSON.stringify(hidePlacementRecord(row()))); + expect('placedEbookFiles' in shown).toBe(false); + expect(shown.audiobook.title).toBe('Book'); + expect(hidePlacementRecord(null)).toBeNull(); + }); + + it.each(['user', 'admin'])('hideStoredPick drops the record for a %s too', (role) => { + const shown = JSON.parse(JSON.stringify(hideStoredPick(row({ selectedTorrent: { title: 'pick' } }), role))); + expect('placedEbookFiles' in shown).toBe(false); + expect('selectedTorrent' in shown).toBe(role === 'admin'); + }); +}); diff --git a/tests/services/request-delete.ebook-placement.test.ts b/tests/services/request-delete.ebook-placement.test.ts index bb0750894..a086f36e5 100644 --- a/tests/services/request-delete.ebook-placement.test.ts +++ b/tests/services/request-delete.ebook-placement.test.ts @@ -23,8 +23,11 @@ vi.mock('@/lib/services/audiobookshelf/api', () => absMock); const { deleteRequest } = await import('@/lib/services/request-delete.service'); -const LEGACY = 'this request was imported before RMAB kept a record of the ebook files it places, so no ebook file was removed'; -const ALREADY_THERE = 'the ebook was already in the library when this request imported it, so it was not removed'; +const NO_RECORD_WHY = 'it was imported before RMAB kept one, or the record could not be saved'; +const LEGACY = `RMAB has no record of the ebook files this request placed (${NO_RECORD_WHY}), so no ebook file was removed`; +const EARLIER = `an earlier import of this request has no record (${NO_RECORD_WHY}), so its ebook file was not removed`; +const ALREADY_THERE = 'RMAB has no record that this request placed the ebook (its import found it already in the library), so it was not removed'; +const MARKER = { earlierImport: 'unrecorded' }; let lib: Library; let otherBooks: Array<{ id: string; title: string; filePath: string | null }>; @@ -218,3 +221,79 @@ describe('#497: an ebook request delete removes only the ebook file that request expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(true); }); }); + +describe('#497: "another request recorded this file" means the same file, not the same spelling', () => { + /** Book.epub in a real title folder, and a second spelling of that folder (a symlinked folder). */ + function twoSpellings() { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub']); + const alias = path.join(lib.media, 'Alias', 'Book'); + fs.mkdirSync(path.dirname(alias), { recursive: true }); + fs.symlinkSync(dir, alias, 'junction'); + const entry = entryFor(path.join(dir, 'Book.epub')); + return { dir, entry, aliasEntry: { ...entry, path: path.join(alias, 'Book.epub') } }; + } + + it.each(['the other request', 'this request'])('keeps the file when %s recorded it through a symlinked folder', async (via) => { + const { dir, entry, aliasEntry } = twoSpellings(); + otherRecords = [{ placedEbookFiles: [via === 'the other request' ? aliasEntry : entry] }]; + load(ebookRequest(book(dir), { placedEbookFiles: [via === 'the other request' ? entry : aliasEntry] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe('Book.epub is also the ebook of another request'); + expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(true); + }); + + it('a stale record of another request at the same path (other bytes) does not keep this file', async () => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub']); + const entry = entryFor(path.join(dir, 'Book.epub')); + otherRecords = [{ placedEbookFiles: [{ ...entry, sha256: 'a'.repeat(64) }] }]; + load(ebookRequest(book(dir), { placedEbookFiles: [entry] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(true); + expect(result.filesKeptReason).toBeUndefined(); + expect(fs.existsSync(path.join(dir, 'Book.epub'))).toBe(false); + }); +}); + +describe('#497: kept reasons', () => { + it('a request delivered before its first record (a pre-upgrade import, then a Manual Import of another format): the new file goes, the old one stays, with the reason', async () => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.pdf', 'Book.epub']); + load(ebookRequest(book(dir), { placedEbookFiles: [MARKER, entryFor(path.join(dir, 'Book.epub'))] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(true); + expect(result.filesKeptReason).toBe(EARLIER); + expect(fs.readdirSync(dir).sort()).toEqual(['Book.m4b', 'Book.pdf', 'cover.jpg']); + }); + + it('a re-import of a pre-upgrade request that found its file already there keeps it with the earlier-import reason', async () => { + const dir = imported(lib.media, 'Author/Book', ['Book.m4b'], ['Book.epub']); + const before = snapshot(lib.media); + load(ebookRequest(book(dir), { placedEbookFiles: [MARKER] })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesDeleted).toBe(false); + expect(result.filesKeptReason).toBe(EARLIER); + expect(snapshot(lib.media)).toEqual(before); + }); + + it('two recorded files kept for the same reason: the reason shows once', async () => { + const dir = path.join(lib.media, 'Ebooks'); + write(path.join(dir, 'Book.epub')); + write(path.join(dir, 'Book.pdf')); + otherBooks = [{ id: 'book-2', title: 'Second', filePath: dir }]; + load(ebookRequest(book(dir, { filesHash: null }), { + placedEbookFiles: [entryFor(path.join(dir, 'Book.epub')), entryFor(path.join(dir, 'Book.pdf'))], + })); + + const result = await deleteRequest('req-1', 'admin-1'); + + expect(result.filesKeptReason).toBe('it is also the folder of "Second"'); + expect(fs.readdirSync(dir).sort()).toEqual(['Book.epub', 'Book.pdf']); + }); +}); diff --git a/tests/utils/ebook-placement.test.ts b/tests/utils/ebook-placement.test.ts index 08bbfe813..28b8b3abc 100644 --- a/tests/utils/ebook-placement.test.ts +++ b/tests/utils/ebook-placement.test.ts @@ -16,7 +16,7 @@ import { createPrismaMock } from '../helpers/prisma'; const prismaMock = createPrismaMock(); vi.mock('@/lib/db', () => ({ prisma: prismaMock })); -const { mergePlacements, placementEntry, readPlacements, recordEbookPlacement, otherRecordedPaths } = await import('@/lib/utils/ebook-placement'); +const { mergePlacements, placementEntry, readPlacements, recordEbookPlacement, otherRecordedEntries, UNRECORDED_EARLIER_IMPORT } = await import('@/lib/utils/ebook-placement'); const sha = (data: string) => createHash('sha256').update(data).digest('hex'); const logger = { info: vi.fn(), warn: vi.fn() } as unknown as Parameters<typeof recordEbookPlacement>[2]; @@ -44,12 +44,12 @@ describe('readPlacements', () => { }); it('[] is a record with no entries', () => { - expect(readPlacements([])).toEqual({ entries: [], unreadable: 0 }); + expect(readPlacements([])).toEqual({ entries: [], unreadable: 0, unrecordedEarlier: false }); }); it('keeps well-formed entries and counts the rest', () => { const ok = good(); - expect(readPlacements([ok, { ...ok, path: 'relative/Book.epub' }, 'x', null])).toEqual({ entries: [ok], unreadable: 3 }); + expect(readPlacements([ok, { ...ok, path: 'relative/Book.epub' }, 'x', null])).toEqual({ entries: [ok], unreadable: 3, unrecordedEarlier: false }); }); it.each([ @@ -63,7 +63,7 @@ describe('readPlacements', () => { ['a negative size', { size: -1 }], ['an unsafe size', { size: 2 ** 53 }], ])('rejects %s', (_label, change) => { - expect(readPlacements([{ ...good(), ...change }])).toEqual({ entries: [], unreadable: 1 }); + expect(readPlacements([{ ...good(), ...change }])).toEqual({ entries: [], unreadable: 1, unrecordedEarlier: false }); }); it('accepts every organizer ebook format, any case', () => { @@ -72,8 +72,14 @@ describe('readPlacements', () => { } }); + it('the earlier-import marker is neither an entry nor unreadable', () => { + const ok = good(); + expect(readPlacements([{ ...UNRECORDED_EARLIER_IMPORT }, ok])).toEqual({ entries: [ok], unreadable: 0, unrecordedEarlier: true }); + expect(readPlacements([{ ...UNRECORDED_EARLIER_IMPORT, path: ok.path }])?.entries).toEqual([]); + }); + it('a non-array value is one unreadable item', () => { - expect(readPlacements({ path: '/x.epub' })).toEqual({ entries: [], unreadable: 1 }); + expect(readPlacements({ path: '/x.epub' })).toEqual({ entries: [], unreadable: 1, unrecordedEarlier: false }); }); }); @@ -104,6 +110,12 @@ describe('mergePlacements', () => { expect(mergePlacements([a, b], newer)).toEqual([b, newer]); }); + it('NULL from a request that delivered before: the record starts with the earlier-import marker', () => { + expect(mergePlacements(null, a, true)).toEqual([UNRECORDED_EARLIER_IMPORT, a]); + expect(mergePlacements(undefined, null, true)).toEqual([UNRECORDED_EARLIER_IMPORT]); + expect(mergePlacements([], a, true)).toEqual([a]); + }); + it('keeps whatever was stored, readable or not', () => { expect(mergePlacements([a, 'junk'], b)).toEqual([a, 'junk', b]); expect(mergePlacements({ odd: true }, b)).toEqual([{ odd: true }, b]); @@ -124,7 +136,7 @@ describe('recordEbookPlacement', () => { await recordEbookPlacement('req-1', record, logger); - expect(prismaMock.request.findUnique).toHaveBeenCalledWith({ where: { id: 'req-1' }, select: { placedEbookFiles: true } }); + expect(prismaMock.request.findUnique).toHaveBeenCalledWith({ where: { id: 'req-1' }, select: { placedEbookFiles: true, completedAt: true } }); expect(prismaMock.request.update).toHaveBeenCalledWith({ where: { id: 'req-1' }, data: { placedEbookFiles: [earlier, { path: record!.target, size: 6, sha256: sha('placed'), placedAt: expect.any(String) }] }, @@ -139,6 +151,26 @@ describe('recordEbookPlacement', () => { expect(prismaMock.request.update).toHaveBeenCalledWith({ where: { id: 'req-1' }, data: { placedEbookFiles: [] } }); }); + it('no record yet but an earlier delivery (completedAt set before this import): the marker goes first', async () => { + prismaMock.request.findUnique.mockResolvedValue({ placedEbookFiles: null, completedAt: new Date('2026-01-01T00:00:00Z') }); + const record = placed('copied'); + + await recordEbookPlacement('req-1', record, logger); + + expect(prismaMock.request.update).toHaveBeenCalledWith({ + where: { id: 'req-1' }, + data: { placedEbookFiles: [UNRECORDED_EARLIER_IMPORT, expect.objectContaining({ path: record!.target })] }, + }); + }); + + it('a record that exists is never marked, whatever completedAt says', async () => { + prismaMock.request.findUnique.mockResolvedValue({ placedEbookFiles: [], completedAt: new Date('2026-01-01T00:00:00Z') }); + + await recordEbookPlacement('req-1', placed('identical'), logger); + + expect(prismaMock.request.update).toHaveBeenCalledWith({ where: { id: 'req-1' }, data: { placedEbookFiles: [] } }); + }); + it('never throws: a database failure is logged and the import goes on', async () => { prismaMock.request.findUnique.mockRejectedValue(new Error('db down')); @@ -157,14 +189,12 @@ describe('recordEbookPlacement', () => { }); }); -describe('otherRecordedPaths', () => { +describe('otherRecordedEntries', () => { it('collects readable entries of other active ebook requests, of any book', async () => { const entry = { path: path.join(root, 'A.epub'), size: 1, sha256: 'a'.repeat(64), placedAt: 't' }; prismaMock.request.findMany.mockResolvedValue([{ placedEbookFiles: [entry, 'junk'] }, { placedEbookFiles: null }]); - const paths = await otherRecordedPaths('req-1'); - - expect([...paths]).toEqual([path.resolve(entry.path)]); + expect(await otherRecordedEntries('req-1')).toEqual([entry]); expect(prismaMock.request.findMany).toHaveBeenCalledWith({ where: { id: { not: 'req-1' }, deletedAt: null, type: 'ebook' }, select: { placedEbookFiles: true }, From d51f74992542ecdd1f6b1c6fa982a22f22efac45 Mon Sep 17 00:00:00 2001 From: kikootwo <folcards@yahoo.com> Date: Mon, 2 Nov 2026 10:55:35 -0500 Subject: [PATCH 3/3] docs: the #497 Limits heading says which limit removes a file Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- documentation/admin-features/request-deletion.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/admin-features/request-deletion.md b/documentation/admin-features/request-deletion.md index 626c527d8..4d7ea89fd 100644 --- a/documentation/admin-features/request-deletion.md +++ b/documentation/admin-features/request-deletion.md @@ -325,7 +325,7 @@ where: { - **Cause:** nothing recorded which file an ebook import placed; `planEbookDelete` guessed from the folder - **Fix:** the ebook organize records the file it copied (`Request.placedEbookFiles`, only for placeFile `copied`); the delete removes only recorded, unchanged files behind rules a-c (D2 above). `planEbookDelete` removed - **Responses:** the record holds absolute library paths and is read only by the delete: every response that returns a request row leaves it out, for admins too (`hidePlacementRecord()`/`hideStoredPick()` in `src/lib/services/request-detail-query.ts`; `GET /api/requests` sets it `undefined` as it does `audiobook.filePath`) -- **Limits** (each keeps a file; none removes more): +- **Limits** (none removes more than before #497; all but the second keep the file): - Ebook requests imported before this version have no record: their delete removes no ebook file (reason shown; remove by hand). A crash between the copy and the record write, or a failed record write, leaves that file unrecorded (kept) - Another active ebook request on the same book whose import found this file already there (`[]`) does not hold it: deleting the request that placed it removes it, and the other request still shows `downloaded` (as before #497) - A delete while the request's import is still running: the file that import places afterwards is not recorded on a live request, so RMAB never removes it