feat(scan): add lance_namespace_query_table setting to opt out of query_table scans

[yichenw]

Problem

Since #232, scans of REST-namespace tables unconditionally go through the Lance Namespace query_table API. Namespace servers that implement the describe/list surface but not query_table now fail every scan with a hard error:

IO Error: Failed to create Lance namespace query_table stream (Lance error: namespace query_table:
Namespace error: Internal error: Failed to parse error response: status=404 Not Found, ...)

We hit this in production: our self-hosted namespace server implements table resolution (describe_table) but not query_table, and the DuckDB engine reads the underlying datasets from object storage directly. After picking up #232 (via the v9 bump in #234), every scan-shaped query against the catalog failed; only metadata-only queries survived.

Change

Add a lance_namespace_query_table session setting, default true (current behaviour unchanged). When set to false, REST-namespace table scans skip the query_table branch in LanceTableEntry::GetScanFunction and open the underlying dataset directly — the pre-#232 path, which the search and maintenance flows still use. Table resolution stays on the namespace; reads then require direct storage access (as documented).

• register the option in LanceExtension::Load (mirrors lance_deferred_materialization)
• gate the namespace-query branch in GetScanFunction on the setting
• document the setting in docs/rest.md and docs/sql.md
• add test/sql/namespace_rest_scan_direct.test: asserts the EXPLAIN Lance Scan Backend flips between namespace_query_table and dataset with the setting, and that scans work in both modes

Notes

• Marked draft to let CI validate the build + namespace test suite first.
• A possible follow-up (happy to discuss): automatic fallback to the direct path when the namespace returns "unsupported operation"/404 for query_table, cached per attached namespace — the setting here is the minimal explicit escape hatch.

[yichenw]

Friendly bump — CI has been green on this for two weeks and it's blocking us from picking up the lance v9 decoder fix (our namespace server doesn't implement query_table, so the current unconditional routing breaks every scan for us). Happy to adjust the approach if you'd prefer an automatic fallback over a setting.