Skip to content

fix(search): send qualified table ids as multi-segment namespace paths in query_table - #228

Open
hellower wants to merge 2 commits into
lance-format:mainfrom
hellower:fix/query-multilevel-table-id
Open

fix(search): send qualified table ids as multi-segment namespace paths in query_table#228
hellower wants to merge 2 commits into
lance-format:mainfrom
hellower:fix/query-multilevel-table-id

Conversation

@hellower

Copy link
Copy Markdown

Problem

apply_base_request in rust/ffi/query_table.rs builds request.id = Some(vec![config.table_id.clone()]) — always a single segment, even for a qualified id like parent$child$tbl. Both namespace search paths (vector and full-text) flow through this one function: src/lance_search.cpp passes cfg.table_id plus the configured delimiter into LanceNamespaceSearchConfig.

For most REST namespace operations a single pre-joined segment is harmless: GET-style operations such as list_tables join the id segments with the delimiter into the URL path, so one segment and a pre-split array are wire-identical. query_table is different: the REST client POSTs the request as JSON and the id array goes into the request body verbatim (lance-namespace-impls 6.0.0 rest.rs::query_table serializes the whole request via .json(&request)). The segment count is therefore server-observable: a vector or FTS search on a multi-level table is sent as one segment and rejected by servers expecting three, even though describe/open/DDL were fixed by #220.

This is a remnant of the id-splitting defect family from #217: #220 fixed the describe/open/DDL paths but did not cover query_table.

Fix

Split the table id on the configured delimiter (default $, matching the REST client) for the REST namespace backend. Directory-namespace ids deliberately stay single-segment: directory namespaces are single-level and a directory name may legitimately contain $.

Also adds rust/ffi/query_table.rs to RUST_FFI_DEPENDS in CMakeLists.txt — the hand-maintained dependency list predates this file, so incremental builds would not otherwise pick up this fix (or any other edit to the file).

Test

Wire-level Rust test with a minimal self-contained in-process HTTP listener: the mock only answers requests whose POSTed JSON body carries the split 3-segment id array (anything else gets a spec-shaped TableNotFound error body), and the test asserts both the percent-encoded joined id in the URL path and the split id array in the POST body, with the default $ delimiter driving the split.

Context

Found while porting the #225 review fixes to a downstream deployment; this defect exists on main independently of #225. Refs #217, #220.

…s in query_table

apply_base_request always built request.id as a single segment. Unlike
GET-style operations (list_tables), whose URL path re-joins the id with
the delimiter and is wire-identical either way, query_table POSTs the id
array in the JSON request body, so the segment count is server-observable:
a vector or full-text search on a multi-level table (parent$child$tbl)
was sent as one segment and rejected by servers expecting three, even
though describe/open/DDL were fixed by lance-format#220.

Split the table id on the configured delimiter (default "$") for the
REST namespace backend. Directory-namespace ids stay single-segment:
directory namespaces are single-level and a directory name may
legitimately contain '$'.

Adds a wire-level test with a minimal in-process mock listener that only
answers for the split 3-segment id and asserts the POSTed body id array.
Also registers rust/ffi/query_table.rs in RUST_FFI_DEPENDS so incremental
builds pick up edits to the file.

Refs lance-format#217, lance-format#220.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3983a7bfd2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rust/ffi/query_table.rs
Document the lifetime and ownership invariants for the three unsafe
blocks in the multi-level id wire test: the FFI call over the
stack-allocated repr(C) config/options, the borrow of the transferred
error-message allocation, and its single reclamation, per
docs/rust_guidelines.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant