Skip to content

fix: give Prowlarr searches a 180 s timeout and pace the nightly retry sweep - #608

Merged
kikootwo merged 3 commits into
mainfrom
fix/601-search-timeout-pacing
Nov 16, 2026
Merged

kikootwo merged 3 commits into
mainfrom
fix/601-search-timeout-pacing

Conversation

@kikootwo

@kikootwo kikootwo commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

What changed

  • Search timeout. A Prowlarr search now has its own 180 s timeout (PROWLARR_SEARCH_TIMEOUT, next to the existing DOWNLOAD_CLIENT_TIMEOUT). It is set per request on the /search GET in ProwlarrService.search(), which every search goes through: the automatic audiobook and ebook search jobs and Interactive Search (audiobook and ebook, from a request or from a book page before a request exists). /indexer, /health, /indexerstats, RSS, .torrent fetches and the download clients keep 60 s.
  • Paced retry sweep. "Retry missing torrents" used to queue up to 50 searches in the same tick, and Bull runs about 28 jobs of the one queue at once, so a nightly run put roughly 28 searches (each hitting every indexer) in flight together. The sweep now queues each search with a Bull delay of floor(n / 2) x 60 s for the nth search it queues, audiobook and ebook on one schedule: at most 2 start a minute, and 50 spread over about 25 minutes. addSearchJob and addSearchEbookJob take an optional delay; every other caller adds none, so their searches start as before. No new setting, no change to queue concurrency.
  • Ebook search refuses a request already past search. The ebook search job wrote searching without checking the request first. Pacing widens the time a queued search waits from seconds to up to about 25 minutes, and meanwhile the RSS monitor, fetch-ebook or Manual Search can start another search and a download. So the guard is part of this fix: like the audiobook search job, the ebook job's searching write now refuses a request that is cancelled, deleted, or already downloading, processing, downloaded, available, awaiting_approval or denied (SEARCH_JOB_REFUSED_STATUSES), and skips with success ("Request is downloading, search skipped"): no second download record, no second download. Its no-result awaiting_search write, an empty re-check just before each download hand-off (Anna's Archive and indexer) and its failure write use the same condition, so a download or cancel that lands during the search is not overwritten and no second download is queued. Every caller queues the job from pending, failed, awaiting_search or awaiting_release, so they are unaffected.
  • While a delayed search waits, its row on the admin logs page shows pending with the time it was queued; it turns active when it starts. Cancelling the job there removes the delayed Bull job as before.
  • Docs: jobs.md, prowlarr.md (the old "30s per search" line was stale), scheduler.md, ebook-sidecar.md.

Why

The search shared the 60 s ceiling sized for .torrent fetches, and the sweep burst made slow indexers slower still. Thanks to joeg-git for the measurements in the issue: with 8 indexers, 8 of 20 searches went past 60 s, 210 "timeout of 60000ms exceeded" errors over 72 hours touched 101 of 140 waiting requests, and 180 s ran for months without a false abort.

Checks

  • Tests written first, failing on main: 50 waiting requests queue in request order with delays 0, 0, 60 s, 60 s, ... 24 min and never more than 2 in any 60 s window; ebook searches share the schedule; skipped requests use no slot; other callers add no delay (Bull options carry delay only when set); the search GET carries 180 s while getIndexers and RSS keep 60 s; a cancelled request's ebook search writes nothing and searches nothing; a delayed ebook search for a request now downloading or available leaves the row alone and queues no download, while a pending one still searches; a download that starts during an ebook search that finds nothing keeps its status; a download that starts during an ebook search that finds a result (Anna's Archive or indexer) keeps its status and gets no second download record or download job, while a pending request still hands off once; a search that throws after a download started leaves the row downloading.
  • npx vitest related --run passed on each commit (on the last one: 157 files, 1959 tests). npx tsc --noEmit clean. eslint: no new findings on the changed files.

Risk and rollback

  • One stalled indexer can make an Interactive Search wait about 6 minutes instead of failing at 60 s: each query waits up to 180 s, and searchWithVariations runs at least 2 queries in sequence.
  • The nightly sweep takes about 25 minutes to start its last search instead of starting all at once; delayed jobs live in Redis and survive a restart.
  • Rollback: revert this commit. No schema or settings change.

Closes #601

🤖 Generated with Claude Code

kikootwo and others added 3 commits November 15, 2026 18:58
…y sweep

A Prowlarr search now has its own 180 s timeout, set per request on the /search GET that every automatic and interactive search goes through. /indexer, /health, /indexerstats, RSS, .torrent fetches and the download clients keep 60 s.

The nightly retry sweep queued up to 50 searches at once, and Bull runs about 28 jobs of the queue together. It now queues each search with a Bull delay of floor(n / 2) x 60 s, audiobook and ebook on one schedule, so at most 2 start a minute. addSearchJob and addSearchEbookJob take an optional delay; other callers add none.

Pacing lets a queued search wait up to about 25 minutes, so the ebook search job's first write now uses the stop guard and skips a cancelled or deleted request, as the audiobook search already does.

Thanks to joeg-git for the timing measurements.

Closes #601

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A paced ebook search can wait in the queue for up to about 25 minutes
while the RSS monitor, fetch-ebook or Manual Search starts another
search and a download. Like the audiobook search job, the ebook job's
searching write now refuses a request that is cancelled, deleted or in
SEARCH_JOB_REFUSED_STATUSES and skips with success, so it adds no second
download record or download. Its no-result awaiting_search write uses
the same condition, so a download or cancel during the search is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A delayed ebook search could claim a request while another search on it
was still running. When that other search started a download first, this
job still created a second download record and queued a second download,
and a throw after the request moved on wrote failed over it.

Both download hand-offs (Anna's Archive and indexer) now start with the
same empty conditional write the audiobook search uses before its grab;
a request cancelled, deleted or past searching returns success with "no
download queued" and nothing is written or queued. The failed write in
the catch uses the same condition.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kikootwo
kikootwo marked this pull request as ready for review November 16, 2026 00:36
@kikootwo
kikootwo merged commit a67b169 into main Nov 16, 2026
1 check passed
@kikootwo
kikootwo deleted the fix/601-search-timeout-pacing branch November 16, 2026 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prowlarr search timeout is a hardcoded 60 s shared with the download-client timeout, and the retry sweep enqueues up to 50 searches at once

1 participant