feat(java): make a declined pushdown visible instead of silent - #972
feat(java): make a declined pushdown visible instead of silent#972keksmd wants to merge 2 commits into
Conversation
|
cc @SemyonSinchenko @SYaoJun — stacked on #971 (shows its commits until that merges, diff will shrink automatically). Green CI. |
|
@SemyonSinchenko @SYaoJun this one builds on #971, so please review that first. It is opened against The slice itself makes a declined pushdown observable: a backend reports which hints it applied, and declining one must not change the results. |
A physical read is not only a URI. A caller knows which columns it needs, which source rows, which values, and how many rows are enough - and a backend that learns all four up front can skip work that a caller would otherwise have to discard after the fact. Add the request half of the read API: an ordered projection, a half-open row range, an AND conjunction of column filters over immutable scalars, and a row limit. A request also reports which of those hints it actually carries, so a backend can answer for each one instead of guessing. Constraint: filter literals stay immutable and lossless. NaN, infinity, and sub-millisecond timestamps are refused rather than silently compared against a value a stored format cannot represent.
A backend that cannot apply a hint must still return the same rows as one that can. That leaves a caller unable to tell an applied filter from an ignored one, which matters: an ignored row range or limit changes cost, not results. Add the reader half of the read API. A read returns a cursor together with a report that names every requested hint as applied or declined, and the result refuses to be built unless the report accounts for exactly the hints the request carried - no gaps, no invented ones. Directive: declining a pushdown must not change the rows or the schema a read returns. The contract test asserts that a reader declining everything produces the same output as one applying everything.
301fc00 to
a91cefa
Compare
|
Rebased onto current This one sits on top of #971 — reviewing and merging #971 first leaves a single commit here. Together the two are the last piece the Parquet backend in #964 needs, so I am holding that PR until both land rather than opening a dependent chain. |
|
@SemyonSinchenko this is the other half of the read contract — a backend reports which hints it actually applied, so a declined pushdown is visible rather than silently changing nothing. Green and rebased on current |
1 participant
Adds the reader side:
PhysicalReader,ReadReport,ReadResult, plus a contract test that pins the invariant a physical reader must hold — a reader that declines every pushdown hint returns the same rows and the same schema as one that applies all of them. Only the reportedapplied/declinedsets differ.Builds on #971 (this PR's base branch). Together with the merged #961 (batch/write shapes) and #971 (request shapes), this completes the read half of the IO API.
Focused verify:
mvn --no-transfer-progress -f maven-projects/pom.xml -pl io-api -am spotless:checkandclean verify -Dspotless.check.skip=true— clean, 63 tests green.