Skip to content

fix: a single-file download reads the file only as fast as the client reads it - #533

Merged
kikootwo merged 1 commit into
mainfrom
reland/510-single-file-backpressure
Nov 2, 2026
Merged

kikootwo merged 1 commit into
mainfrom
reland/510-single-file-backpressure

Conversation

@kikootwo

@kikootwo kikootwo commented Nov 2, 2026

Copy link
Copy Markdown
Owner

Summary

A single-file download now reads the file only as fast as the client reads it. Before, the route queued every chunk it read from disk for the client, so a paused or slow client made the server read the whole file into memory. The multi-file zip already worked this way (#490); this does the same for one file.

Changes

  • src/app/api/requests/[id]/download/route.ts: the zip's pull loop moves into a shared pullFrom helper, and the single-file branch uses it through a new streamFile. The next chunk is read only when the client takes the last one.
  • A cancel destroys the file stream and closes the file, with no error logged. A read error is logged once, now with the request id, and ends the response with that error.
  • Headers (Content-Type, Content-Disposition, Content-Length) and the token and account checks are unchanged.
  • documentation/backend/services/auth.md: new "Request File Download Link" section (both branches stream with backpressure) and a Fixed Issues entry for Single-file request download reads the whole file into memory when the client stops reading #500. TABLEOFCONTENTS.md links to it.

Tests

New tests/api/requests-download.file-stream.test.ts (4 tests, real route):

  • A paused client: a source that makes each chunk on demand has made at most 4 of 64 chunks after the client reads one; reading on delivers every chunk in order.
  • A real 32 MiB file: under 1 MiB read from disk while the client pauses; reading resumes when the client does.
  • A cancel closes the file and destroys the stream, with no error logged and nothing uncaught.
  • A read error (EIO) ends the response with that error, is logged once with the request id, and the file is closed.

Against main's route, 3 of the 4 fail: both slow-reader tests, and the read-error test only on the request id now in the log line. The cancel test passes on main.

Checks

  • npx vitest related --run on the route and the new test: 4 files, 21 passed, 2 skipped (Windows-only skips in the zip tests).
  • npx tsc --noEmit: clean. npx eslint on the changed files: 0 errors, 0 warnings.
  • On node:24-bookworm (Node v24.22.2, the image's base): a client that reads one chunk of a 64 MiB file and pauses for 1 s leaves 192 KiB read from disk (main's code: all 64 MiB), and a cancel leaves the stream destroyed and closed.

Risk and rollback

Low. Only the single-file response body changes, and it now uses the same streaming code the zip has used since #490. Rollback is a revert of this commit.

Closes #500

Based on #510 by @pintu544.

🤖 Generated with Claude Code

… reads it (#500)

The single-file branch of GET /api/requests/:id/download queued every chunk
read from disk for the client, so a paused or slow client made the server
read the whole file into memory. It now uses the same pull-based stream as
the multi-file zip (#490): one shared pullFrom helper reads the next chunk
only when the client takes the last one. A cancel destroys the file stream
without logging an error; a read error is logged with the request id and
ends the response with that error. Headers unchanged.

Based on #510 by @pintu544.

Co-authored-by: pintu544 <64580363+pintu544@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kikootwo
kikootwo marked this pull request as ready for review November 2, 2026 17:17
@kikootwo
kikootwo merged commit 11b88c4 into main Nov 2, 2026
1 check passed
@kikootwo
kikootwo deleted the reland/510-single-file-backpressure branch November 2, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Single-file request download reads the whole file into memory when the client stops reading

1 participant