fix: reuse and close executor namespace clients

[hfutatzhanghb]

Summary

• create the executor-side namespace client once per LanceColumnarPartitionReader instead of once per fragment
• reuse the task-owned namespace across every fragment in the input partition
• close AutoCloseable namespace implementations after fragment resources on normal completion and scan/open failures
• add regression coverage for multi-fragment reuse, idempotent cleanup, failure cleanup, and the disabled-refresh path

Root cause

With executor_credential_refresh=true, LanceFragmentScanner.create() called
LanceRuntime.getOrCreateNamespace() before every fragment. Despite its name, that method
always creates a new LanceNamespace connection. The fragment scanner owned and closed its
native scanner and dataset, but nobody owned or closed the namespace client.

Hive2 and Hive3 namespace implementations are Closeable and own HMS client pools. As a
result, a Spark task scanning multiple fragments repeatedly created HMS-backed namespace
clients and leaked their connections.

Fix

LanceColumnarPartitionReader now owns executor namespace initialization and cleanup because
its lifecycle matches one Spark task. Initialization remains lazy and preserves the existing
executor_credential_refresh and useNamespaceOnWorkers gates. The namespace stays attached
to the task-local read options while fragments are scanned, then is cleared and closed after
the active fragment scanner.

The failure path preserves the original exception and attaches cleanup failures as suppressed
exceptions. Repeated reader close calls remain idempotent.

Validation

Regression tests were added against the existing real local Lance test dataset with a
recording namespace implementation. They cover:

• one namespace initialization across a two-fragment task
• exactly one namespace close, including repeated reader close
• namespace cleanup when fragment open fails
• no namespace initialization when executor_credential_refresh=false

Local Maven tests, compilation, lint, and formatting checks were intentionally not run under
the repository-specific Lance workflow policy; GitHub Actions is the validation gate.

[hfutatzhanghb]

Hi @LuciferYang ~ could you please help review this PR when have free time? Thanks very much !!!