Skip to content

fix: a multi-file request download streams as it is read and closes its files when cancelled - #498

Merged
kikootwo merged 1 commit into
mainfrom
fix/490-zip-stream
Nov 2, 2026
Merged

kikootwo merged 1 commit into
mainfrom
fix/490-zip-stream

Conversation

@kikootwo

Copy link
Copy Markdown
Owner

Closes #490

What changed

  • Multi-file request download (GET /api/requests/:id/download, src/app/api/requests/[id]/download/route.ts): the zip is built by a new streamZip() in the route.
    • Pull-based response: a ReadableStream whose pull() reads the next chunk of the archive only when the client has taken the last one. A slow or paused client holds the archive back instead of the server queueing the zip in memory.
    • One file at a time: archiver starts reading a stream as soon as it is appended, so each file is appended only after the previous entry is written. Only the file being zipped is open.
    • Cancel: the archive is aborted and the file being read is closed.
    • Failure: an archive error, an archiver warning, or a read error on a file is logged once (Zip download failed, with the request id and the error message) and ends the response with an error.
    • Each file is stat'ed before the zip starts, and its entry gets the same name, mode and date as before. Directories and symlinks still go through archive.file(), which opens no file for them.
  • Single-file downloads, the account check (fix: download links check the account like a sign-in does #484), entry names (flat, /, \ and : become _), entry order and compression level 6 are unchanged.
  • Docs: documentation/backend/services/auth.md (new Fixed Issues entry).

Why

The old response wrapped the archive in a ReadableStream whose start() enqueued every data event:

  • It never checked desiredSize, so a client that stopped reading made the server read the files and queue the zip in memory.
  • cancel() only called archive.abort(), which does not close the file being read. The data listener kept enqueueing into the closed stream and threw ERR_INVALID_STATE: Invalid state: Controller is already closed, uncaught.
  • The archive had no error listener, so a read error on one file was uncaught and the response never ended.

Evidence

  • tests/api/requests-download.zip-stream.test.ts runs the real route and real archiver over real temp files, counting the files opened, closed and bytes read through fs.open, fs.read and fs.close:
    • Cancel after 512 KB of a 3-file zip: every opened file is closed and nothing is thrown.
    • A read error (EIO) on the second file: the response fails within 5 s, every opened file is closed, nothing is thrown, and one error is logged with the request id.
    • A client that pauses for 3 s on a 100 MB zip (5 random 20 MB files): under 16 MB is read from disk, the zip carries on when the client reads again, and a cancel then closes every file.
  • With main's route (e25f16d) and the new test file: 3 failed, all on an assertion. Cancel leaves 1 file open and throws ERR_INVALID_STATE. The read error times out at 5 s with 1 file open and an uncaught EIO. The paused client let the server read 38.6 MB of 100 MB in 3 s (46.9 and 80.9 MB in earlier runs).
  • On this branch: 3/3. The paused client let the server read 2.2 MB of 100 MB.
  • E2E (our image, built from this branch): smoke 1/1, and 349-zip-download 1/1: Project Hail Mary.zip, 83,300 bytes (the same size as on archiver 8 before this change), 4 flat entries, each matching its file by sha256. No Zip download failed or ERR_INVALID_STATE line in the app log.

Checks

  • Unit tests for the changed code (vitest related on the route plus the branch's test file): PASS, 3 files, 17 passed, 2 skipped (POSIX-only file names, on a Windows host)
  • Type check and lint: PASS (tsc --noEmit clean; ESLint clean on both touched code files)
  • Image build and E2E: PASS (image rmab-e2e/t117:186098f; smoke and 349-zip-download, one fresh stack each)
  • Full unit suite: PR Tests (CI)

Risk and rollback

  • A file that disappears between the folder listing and the zip now fails the download (500 before the zip starts, or the response ends with an error once it has started). Before, archiver skipped it with a warning and sent a zip without it.
  • A download that fails partway now ends with an error instead of hanging, so a browser shows a failed download where it used to wait.
  • Nothing stored changes. Rollback: revert this PR.

Release note

🤖 Generated with Claude Code

…ad error and wait for the reader (#490)

The zip is now pull-based and adds one file at a time, so only the file
being zipped is open and the next bytes are made only when the client
reads. A cancel closes that file; a read error or archiver warning is
logged once with the request id and ends the response with an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kikootwo
kikootwo marked this pull request as ready for review October 27, 2026 12:09
@kikootwo
kikootwo merged commit abf3639 into main Nov 2, 2026
1 check passed
@kikootwo
kikootwo deleted the fix/490-zip-stream branch November 2, 2026 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multi-file request download stalls or keeps a file open when interrupted

1 participant