Repository navigation
fix: a multi-file request download streams as it is read and closes its files when cancelled - #498
Merged
Merged
Conversation
…ad error and wait for the reader (#490) The zip is now pull-based and adds one file at a time, so only the file being zipped is open and the next bytes are made only when the client reads. A cancel closes that file; a read error or archiver warning is logged once with the request id and ends the response with an error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #490
What changed
GET /api/requests/:id/download,src/app/api/requests/[id]/download/route.ts): the zip is built by a newstreamZip()in the route.ReadableStreamwhosepull()reads the next chunk of the archive only when the client has taken the last one. A slow or paused client holds the archive back instead of the server queueing the zip in memory.Zip download failed, with the request id and the error message) and ends the response with an error.archive.file(), which opens no file for them./,\and:become_), entry order and compression level 6 are unchanged.documentation/backend/services/auth.md(new Fixed Issues entry).Why
The old response wrapped the archive in a
ReadableStreamwhosestart()enqueued everydataevent:desiredSize, so a client that stopped reading made the server read the files and queue the zip in memory.cancel()only calledarchive.abort(), which does not close the file being read. Thedatalistener kept enqueueing into the closed stream and threwERR_INVALID_STATE: Invalid state: Controller is already closed, uncaught.errorlistener, so a read error on one file was uncaught and the response never ended.Evidence
tests/api/requests-download.zip-stream.test.tsruns the real route and real archiver over real temp files, counting the files opened, closed and bytes read throughfs.open,fs.readandfs.close:EIO) on the second file: the response fails within 5 s, every opened file is closed, nothing is thrown, and one error is logged with the request id.main's route (e25f16d) and the new test file: 3 failed, all on an assertion. Cancel leaves 1 file open and throwsERR_INVALID_STATE. The read error times out at 5 s with 1 file open and an uncaughtEIO. The paused client let the server read 38.6 MB of 100 MB in 3 s (46.9 and 80.9 MB in earlier runs).349-zip-download1/1:Project Hail Mary.zip, 83,300 bytes (the same size as on archiver 8 before this change), 4 flat entries, each matching its file by sha256. NoZip download failedorERR_INVALID_STATEline in the app log.Checks
vitest relatedon the route plus the branch's test file): PASS, 3 files, 17 passed, 2 skipped (POSIX-only file names, on a Windows host)tsc --noEmitclean; ESLint clean on both touched code files)349-zip-download, one fresh stack each)Risk and rollback
Release note
🤖 Generated with Claude Code