feat(#83): read member content from an archive container - #457
Merged
Conversation
Adds `ReadMemberOfFile(path, member, max_bytes)` to the archive reader - the
entry point the content-side predicates need, so `-grep`, `-content` and
`{hash}` can eventually see inside a container instead of only listing it.
Design points, each pinned by a test:
- Streaming, single pass, stopping at the match, so reading an early member of
a huge archive never decompresses the rest. `kBlockSize` moves from a
function-local constant into the file-scope anonymous namespace, shared with
`ListMembersOfFile`.
- Member names are matched NORMALIZED: a leading `./` is ignored on either
side, and so is a trailing `/`. Tar writes the same member several ways
(`dir/x` vs `./dir/x`, and a directory as `dir/`), so neither spelling can be
authoritative. The trailing-slash half matters for the error contract: a
lookup for `dir` now FINDS the directory and reports "no content" rather than
misreporting "no such member".
- Every failure is a distinct status, because a caller has to tell them apart:
`NotFound` (no such member), `FailedPrecondition` (member has no content,
e.g. a directory), `InvalidArgument` (not an archive at all), `DataLoss`
(broken archive) and `ResourceExhausted` (bomb guard).
- The bomb guard bounds the READ LOOP, not the declared size: a small header
can promise a huge expansion, so the header size is used only to `reserve`.
`max_bytes` is inclusive - content exactly at the limit is fine.
The test fixture gains `WriteArchive` / `WriteArchiveWithDirectory`, since the
existing `MakeArchive` is in-memory while this entry point streams from a path.
helly25
enabled auto-merge (squash)
August 11, 2026 01:01
This was referenced Aug 11, 2026
helly25
added a commit
that referenced
this pull request
Aug 11, 2026
Conflict: #455 (archive VFS backend) and this branch each appended targets to extra_modules/archive/BUILD.bazel. Both sets are kept. Also does what this PR promised once #457 landed: `archive_reader.cc` drops its local copy of the member-name normalization and uses the shared `NormalizeMemberName` from the member-path library, so the rule exists once for both readers rather than being duplicated on main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
ReadMemberOfFile(path, member, max_bytes)to the archive reader - the entry point the content-side predicates need, so-grep,-contentand{hash}can eventually see inside a container instead of only listing it.Design points, each pinned by a test:
kBlockSizemoves from a function-local constant into the file-scope anonymous namespace, shared withListMembersOfFile../is ignored on either side, and so is a trailing/. Tar writes the same member several ways (dir/xvs./dir/x, and a directory asdir/), so neither spelling can be authoritative. The trailing-slash half matters for the error contract: a lookup fordirnow FINDS the directory and reports "no content" rather than misreporting "no such member".NotFound(no such member),FailedPrecondition(member has no content, e.g. a directory),InvalidArgument(not an archive at all),DataLoss(broken archive) andResourceExhausted(bomb guard).reserve.max_bytesis inclusive - content exactly at the limit is fine.The test fixture gains
WriteArchive/WriteArchiveWithDirectory, since the existingMakeArchiveis in-memory while this entry point streams from a path.Part of #83. Not armed for auto-merge; queued behind #450 / #455 / #456.