Skip to content

fix(index): remap ZoneMap search results - #8512

Merged
Xuanwo merged 6 commits into
mainfrom
gatekeeper/fix-8504-1
Aug 30, 2026
Merged

fix(index): remap ZoneMap search results#8512
Xuanwo merged 6 commits into
mainfrom
gatekeeper/fix-8504-1

Conversation

@lance-gatefixer

Copy link
Copy Markdown
Contributor

Summary

  • remap ZoneMap selected and nullable row addresses through the loaded fragment-reuse index
  • preserve exact, at-most, and at-least search-result semantics during remapping
  • add a multi-fragment deferred-compaction regression covering null, equality, and range predicates

Root cause

ZoneMap retained physical row addresses from index creation. Deferred-remap compaction moved those rows and supplied a fragment-reuse remapper when the index was loaded, but ZoneMap search never applied it. The exact null bitmap therefore returned stale addresses without a refinement filter and silently dropped live rows.

Validation

  • cargo test -p lance test_read_zonemap_index_with_defer_index_remap -- --nocapture
  • cargo test -p lance-index scalar::zonemap::tests
  • cargo fmt --all -- --check
  • cargo clippy --all --tests --benches -- -D warnings
  • git diff --check

Fixes #8504

@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer bug Something isn't working labels Aug 12, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-changes Latest Gatekeeper recommendation requests changes. label Aug 12, 2026
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@lance-gatefixer

Copy link
Copy Markdown
Contributor Author

Blocked: the current-head linux-coverage-build check is stopped by runner TLS infrastructure before repository code can build.

Remote head 58744773e3412d77b7a20dc68ed77eff916b3aca contains current main tip 67273e058b4860527dec39e7df7c8434650bb8c7; Rust run 31586073678, job 94080088712 failed in rui314/setup-mold because GitHub returned a self-signed dotcom.glb certificate whose hostname does not match github.com. I fetched and verified base ancestry, re-read the current run logs, and confirmed the other Rust build, clippy, formatting, rustdoc, and platform jobs passed, leaving no attributable code repair or safe in-scope workflow change. Please rerun the failed Rust workflow after runner networking is healthy; alternatively, a maintainer can repair or confirm the runner certificate/proxy configuration and start a fresh run.

@lance-gatekeeper lance-gatekeeper Bot removed the K-changes Latest Gatekeeper recommendation requests changes. label Aug 14, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-changes Latest Gatekeeper recommendation requests changes. label Aug 14, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-changes Latest Gatekeeper recommendation requests changes. label Aug 14, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added K-approved Latest Gatekeeper recommendation permits acceptance. and removed K-approved Latest Gatekeeper recommendation permits acceptance. labels Aug 14, 2026
@lance-gatekeeper lance-gatekeeper Bot added K-approved Latest Gatekeeper recommendation permits acceptance. and removed K-approved Latest Gatekeeper recommendation permits acceptance. labels Aug 21, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

All reviewed false-negative paths are now closed. Deferred-remap search and consolidation stay in the live address domain, exact null bitmaps drive transformed counts, and ambiguous legacy singleton statistics fail safely, including Decimal zones with missing extrema.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ZoneMap index drops live rows after deferred-remap compaction

1 participant