Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions documentation/admin-features/request-deletion.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,8 @@ model Request {
(src/lib/utils/seeding-policy.ts, same rule as cleanup_seeded_torrents: every set
limit met; 0 turns a limit off; unreported ratio with ratioLimit > 0 = not met)
IF another active request uses the same torrentHash (otherActiveRequestsUsingTorrent,
src/lib/utils/shared-torrent.ts, shared with the cleanup job):
→ Keep torrent → torrentsKeptShared++
src/lib/utils/shared-torrent.ts), whatever its status, even finished and done seeding:
→ Keep torrent → torrentsKeptShared++ (the cleanup job may remove it later, see below)
ELSE:
→ Delete torrent + files → torrentsRemoved++ → row downloadStatus = 'removed'

Expand Down Expand Up @@ -162,6 +162,13 @@ where: {
- **Soft-deleted requests:** Delete torrent + hard-delete request when seeding complete
- **Unlimited seeding:** Hard-delete orphaned request immediately (no monitoring)
- **Not RMAB's torrent:** never removed; a soft-deleted request is hard-deleted, the torrent stays. Details: [scheduler.md](../backend/services/scheduler.md) (cleanup_seeded_torrents)
- **Shared torrent (#283):** after ownership and this row's seeding check, `otherActiveRequestsSharingTorrent()` lists the other non-deleted requests on the torrent (same hash forms as the delete guard). Each keeps it (logged with the reason) unless `siblingKeepsTorrent()` (`src/lib/utils/shared-torrent.ts`) finds it:
- finished: `FINISHED_REQUESTS` (audiobook `available`, ebook `downloaded`; also the candidate query's statuses). Any other status keeps it (`downloading`, `processing`, audiobook `downloaded`, `failed`, `cancelled`, ...)
- an audiobook also imported by RMAB since it took the torrent: its book record `completed` with a `filePath` (written only by the organizer after the copy) and `completedAt` later than the row's `createdAt`. A library scan can mark it `available` mid-copy, and a library match alone never passes, so either keeps it
- every row it has for the torrent recorded for the configured client type
- its own indexer's requirement met (`evaluateSeeding()` with its row's indexer, by the same probe answer); unlimited or unconfigured indexer keeps it
- None keeps it → one `deleteDownload`, this row `removed` (or soft-deleted: hard-deleted), the siblings' rows `removed` in one `updateMany` (also rows RMAB did not add, which never reach the guard themselves). Same run: `deletedHashes`; later runs: the probe's not-found → `removed`
- Soft-deleted requests never keep it (unchanged)

### API Endpoint

Expand Down Expand Up @@ -288,6 +295,11 @@ where: {
- **Issue:** a release already in the client (added by hand or another app) was grabbed by its existing hash; deleting the request (or the seeding cleanup) then removed it with its files
- **Fix:** `DownloadHistory.addedByRmab` recorded at the grab; delete and cleanup act only on RMAB's torrents (`resolveOwnership()`); rows from before the fix are judged by the client's added time

**6. Torrent Shared by Several Finished Requests Never Cleaned Up (#283)**
- **Issue:** a series pack (or re-request) used by several finished requests stayed in the client forever
- **Cause:** the cleanup job's shared-torrent check counted every non-deleted sibling, so each finished request kept it for the others
- **Fix:** cleanup job only: a sibling keeps it unless finished and done seeding by its own indexer's rule (`siblingKeepsTorrent()`); request deletion unchanged

## File Structure

```
Expand Down
2 changes: 1 addition & 1 deletion documentation/phase3/download-clients.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ finalPath = config.customPath ? path.join(downloadDir, config.customPath) : down
- Bull retry after an add that succeeded (lost response, history insert/enqueue failed, worker died) or two download jobs for one request → RMAB's own torrent recorded `false`
- Admin retry-download (`src/app/api/admin/requests/[id]/retry-download/route.ts`) re-adding a stored link while RMAB's torrent is still there → recorded `false`
- Re-request/replace/series pack of a hash whose owner row was hard-deleted
- A `false` re-request row counts as sharing the torrent (`otherActiveRequestsUsingTorrent`), so cleanup hard-deletes the soft-deleted owner row and keeps the torrent; the `false` row is never cleaned (same "owner row hard-deleted" leak, caused by RMAB's own cleanup)
- A `false` re-request row that is not finished or not done seeding counts as sharing the torrent (#283, [request-deletion.md](../admin-features/request-deletion.md)), so cleanup hard-deletes the soft-deleted owner row and keeps the torrent; the `false` row is never cleaned (same "owner row hard-deleted" leak, caused by RMAB's own cleanup). Finished and done seeding: the owner row removes it and marks the `false` row `removed`
- Switching to a new client instance of the same type: its well-formed "not here" answers mark those rows `removed` for good (nothing clears the mark), so RMAB never cleans them after the torrents are migrated
- Client migrated (added times reset) or without added time (legacy rows); clock drift between client and DB > 5 min
- Deluge loads torrents asynchronously at startup (`load_state` → `add_async`), so a probe during startup can miss a torrent → row marked `removed` (never cleaned, never lost); Transmission 4.0.6 loads all torrents in one session-thread task before RPC answers (no partial list), 3.x not verified
Expand Down
4 changes: 2 additions & 2 deletions documentation/phase3/file-organization.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,8 +271,8 @@ exiftool "audiobook.m4b" | grep -i asin
1. Find requests with status 'available' or soft-deleted (orphaned downloads)
2. Query qBittorrent for actual `seeding_time` field
3. **CRITICAL: Check if torrent hash is shared by other active requests**
- If yes → Skip torrent deletion, only hard-delete the soft-deleted request record
- If no → Delete torrent + files
- Any of them not finished, or not done seeding by its own indexer (#283) → Skip torrent deletion, only hard-delete the soft-deleted request record
- Else → Delete torrent + files, mark every sharing row removed ([request-deletion.md](../admin-features/request-deletion.md))
4. Delete torrent + files only after seeding requirement met
5. Respects config (0 = never cleanup)

Expand Down
35 changes: 23 additions & 12 deletions src/lib/processors/cleanup-seeded-torrents.processor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import { prisma } from '../db';
import { RMABLogger } from '../utils/logger';
import { CLIENT_PROTOCOL_MAP, DownloadClientType, IDownloadClient } from '../interfaces/download-client.interface';
import { evaluateSeeding, isUnlimitedSeeding } from '../utils/seeding-policy';
import { otherActiveRequestsUsingTorrent } from '../utils/shared-torrent';
import { FINISHED_REQUESTS, otherActiveRequestsSharingTorrent, siblingKeepsTorrent } from '../utils/shared-torrent';
import { createIndexerConfigLookup, describeIgnoredIndexerId, parseSavedIndexers } from '../utils/indexer-config';
import { knownNotOwned, resolveOwnership } from '../utils/torrent-ownership';

Expand All @@ -28,16 +28,14 @@ const MAX_PROBE_ERRORS_IN_A_ROW = 3;

// Completed requests + soft-deleted requests (orphaned downloads), each with a finished selected download.
// IMPORTANT: Only cleanup requests that are truly complete and not being actively processed.
// NOTE: Multiple requests can share the same torrent hash (e.g., re-requesting same audiobook);
// before deleting a torrent we check if other active requests are using it.
// NOTE: Multiple requests can share the same torrent hash (e.g., re-requesting same audiobook, a series pack);
// a torrent is deleted only when every other active request using it is finished and done seeding too (#283).
// NOTE: Ebooks downloaded via indexer search use torrent clients and need seeding cleanup too.
// Direct HTTP ebook downloads are naturally skipped (no torrent hash / unknown client type).
const CANDIDATES = {
OR: [
// Audiobook requests that are fully available (matched in Plex/ABS)
{ type: 'audiobook', status: 'available', deletedAt: null },
// Ebook requests that are fully downloaded (terminal state for ebooks)
{ type: 'ebook', status: 'downloaded', deletedAt: null },
// Finished: audiobook available (matched in Plex/ABS), ebook downloaded (terminal state for ebooks)
...FINISHED_REQUESTS.map((finished) => ({ ...finished, deletedAt: null })),
// Soft-deleted requests of any type (orphaned downloads)
{ deletedAt: { not: null } },
],
Expand Down Expand Up @@ -235,10 +233,17 @@ export async function processCleanupSeededTorrents(payload: CleanupSeededTorrent

logger.info(`Download ${downloadInfo.name} (${indexerName}) has met seeding requirement: ${progress}`);

// CRITICAL: Check if any other active (non-deleted) request is using this same download
const otherActiveRequests = await otherActiveRequestsUsingTorrent(request.id, downloadHistory.torrentHash);
if (otherActiveRequests.length > 0) {
logger.info(`Skipping download deletion - ${otherActiveRequests.length} other active request(s) still using this download (IDs: ${otherActiveRequests.map(r => r.id).join(', ')})`);
// CRITICAL: every other active (non-deleted) request using this download must be finished, and
// done seeding by its own indexer's rule (same client answer); any other one keeps it
const siblings = await otherActiveRequestsSharingTorrent(request.id, downloadHistory.torrentHash);
const keeping = siblings.flatMap((sibling) => {
const reason = siblingKeepsTorrent(sibling, clientType, (row) => evaluateSeeding(
lookupIndexer({ indexerId: row.indexerId, indexerName: row.indexerName ?? '', protocol }).config, downloadInfo
));
return reason ? [`${sibling.id} (${reason})`] : [];
});
if (keeping.length > 0) {
logger.info(`Skipping download deletion - ${keeping.length} other active request(s) still using this download: ${keeping.join(', ')}`);

// If this is a soft-deleted request, hard delete it but DON'T delete the download
if (request.deletedAt) {
Expand All @@ -249,9 +254,15 @@ export async function processCleanupSeededTorrents(payload: CleanupSeededTorrent
return 'seeding';
}

// Safe to delete - no other active requests using this download
// Safe to delete - no other active request needs this download
await client.deleteDownload(clientId, true); // true = delete files
deletedHashes.add(clientId.toLowerCase());
if (siblings.length > 0) {
// Their rows say the torrent is gone too (a row RMAB did not add never gets here on its own)
const siblingRowIds = siblings.flatMap((sibling) => sibling.downloadHistory.map((row) => row.id));
await prisma.downloadHistory.updateMany({ where: { id: { in: siblingRowIds } }, data: { downloadStatus: 'removed' } });
logger.info(`Download was shared with ${siblings.length} finished request(s) (${siblings.map((s) => s.id).join(', ')}): marked removed for them too`);
}

// If this is a soft-deleted request (orphaned download), hard delete it now
if (request.deletedAt) {
Expand Down
119 changes: 107 additions & 12 deletions src/lib/utils/shared-torrent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,40 +2,135 @@
* Component: Shared Torrent Guard
* Documentation: documentation/admin-features/request-deletion.md
*
* Several requests can share one torrent (e.g. re-requesting the same book).
* Before a torrent is removed from the client, every caller (scheduled
* seeding cleanup, request deletion) checks that no other active request
* still uses it.
* Several requests can share one torrent (e.g. re-requesting the same book, a series pack).
* Before a torrent is removed from the client, every caller checks the other active requests
* that use it. Request deletion: any of them keeps it. Scheduled seeding cleanup: each of them
* keeps it unless it is finished and its own indexer's seeding requirement is met (#283).
*/

import { prisma } from '../db';
import { torrentHashForms } from '../integrations/torrent-ids';
import type { SeedingEvaluation } from './seeding-policy';

export interface ActiveRequestRef {
id: string;
status: string;
}

/** Finished requests (the seeding cleanup's active candidates): nothing more will read the torrent */
export const FINISHED_REQUESTS = [
{ type: 'audiobook', status: 'available' },
{ type: 'ebook', status: 'downloaded' },
] as const;

/** Another active request using the torrent, with its selected row(s) for that torrent */
export interface TorrentSibling extends ActiveRequestRef {
type: string;
/** Its book record (shared by every request for the book); the organizer completes it after a successful copy */
audiobook: SiblingBookRecord | null;
downloadHistory: TorrentSiblingDownload[];
}

export interface SiblingBookRecord {
status: string;
filePath: string | null;
completedAt: Date | null;
}

export interface TorrentSiblingDownload {
id: string;
indexerId: number | null;
indexerName: string | null;
downloadClient: string | null;
createdAt: Date;
}

function otherActiveRequestsWhere(requestId: string, torrentHash: string) {
return {
id: { not: requestId },
deletedAt: null,
downloadHistory: {
some: {
torrentHash: { in: torrentHashForms(torrentHash) },
selected: true,
},
},
};
}

/**
* Active (not deleted) requests other than `requestId` whose selected
* download is the torrent `torrentHash` (hex and base32 forms, in either letter case, match each other). Empty when there is no hash.
* Request deletion keeps the torrent while this is not empty.
*/
export async function otherActiveRequestsUsingTorrent(
requestId: string,
torrentHash: string | null | undefined
): Promise<ActiveRequestRef[]> {
if (!torrentHash) return [];
return prisma.request.findMany({
where: {
id: { not: requestId },
deletedAt: null,
where: otherActiveRequestsWhere(requestId, torrentHash),
select: { id: true, status: true },
});
}

/** Same requests as `otherActiveRequestsUsingTorrent`, with type and their selected rows for this torrent (seeding cleanup) */
export async function otherActiveRequestsSharingTorrent(
requestId: string,
torrentHash: string | null | undefined
): Promise<TorrentSibling[]> {
if (!torrentHash) return [];
return prisma.request.findMany({
where: otherActiveRequestsWhere(requestId, torrentHash),
select: {
id: true,
status: true,
type: true,
audiobook: { select: { status: true, filePath: true, completedAt: true } },
downloadHistory: {
some: {
torrentHash: { in: torrentHashForms(torrentHash) },
selected: true,
},
where: { torrentHash: { in: torrentHashForms(torrentHash) }, selected: true },
select: { id: true, indexerId: true, indexerName: true, downloadClient: true, createdAt: true },
},
},
select: { id: true, status: true },
});
}

/**
* A library scan can mark an audiobook `available` while its organize is still copying (#283).
* Only the organizer completes the book record with a `filePath`, after the copy; it must have done
* so after this row took the torrent, since the record is shared by every request for the book.
* A book only matched in the library (no RMAB import) never passes.
*/
function importedSince(book: SiblingBookRecord | null, row: TorrentSiblingDownload): boolean {
if (book?.status !== 'completed' || !book.filePath || !book.completedAt) return false;
return new Date(book.completedAt).getTime() > new Date(row.createdAt).getTime();
}

/**
* Seeding cleanup only (#283): why `sibling` still needs the torrent, or null when it lets it go.
* It lets it go only when it is finished (an audiobook also imported by RMAB since it took the
* torrent), every row it has for the torrent was recorded for `clientType`, and its own indexer's
* requirement is met (`seeding(row)`, by the same client answer).
*/
export function siblingKeepsTorrent(
sibling: TorrentSibling,
clientType: string,
seeding: (row: TorrentSiblingDownload) => SeedingEvaluation
): string | null {
if (!FINISHED_REQUESTS.some((f) => f.type === sibling.type && f.status === sibling.status)) {
return `${sibling.type ?? 'request'} ${sibling.status}`;
}
const rows = sibling.downloadHistory ?? [];
if (rows.length === 0) return 'no download row for this torrent';
for (const row of rows) {
const recordedFor = row.downloadClient || 'qbittorrent';
if (recordedFor !== clientType) return `recorded for ${recordedFor}`;
if (sibling.type === 'audiobook' && !importedSince(sibling.audiobook, row)) {
return 'no RMAB import since it took this torrent';
}
const { unlimited, met, progress } = seeding(row);
if (unlimited) return `unlimited seeding (${row.indexerName ?? 'no indexer'})`;
if (!met) return `still seeding for ${row.indexerName ?? 'its indexer'}: ${progress}`;
}
return null;
}
Loading
Loading