feat(java): align inverted index options

[ddupg]

Summary

• align Java tokenizer support with PyLance and the existing Java API documentation by enabling Jieba and Lindera in the Java JNI artifact, with a lightweight Jieba indexing and search test
• expose the remaining Rust inverted-index build options through the Java builder
• preserve source compatibility for the ineffective skipMerge option while deprecating it and stopping serialization of the unsupported field

Test plan

• ./mvnw test (before rebasing onto the latest main)
• cargo fmt --manifest-path java/lance-jni/Cargo.toml --all -- --check
• cargo clippy --tests --manifest-path java/lance-jni/Cargo.toml
• ./mvnw -Dtest=InvertedIndexParamsTest test after the final test cleanup

[lance-gatekeeper]

❌ Gate recommendation: request changes.

The new Java code-analyzer preset currently cannot cross the generic Rust parameter boundary, so it cannot create an index. #8779 independently fixes that sparse-parameter resolution path; merge it first, rebase this PR, and add JNI-backed code-analyzer coverage to verify the preset end to end.

[lance-gatekeeper]

analyzer("code") is serialized without base_tokenizer, but resolved_inverted_params currently overlays that sparse JSON onto the full text defaults, retaining base_tokenizer="simple". The newly exposed Java preset therefore cannot create an index. #8779 removes that eager default merge and is an independently mergeable prerequisite; after it lands, please rebase this PR and add this JNI-backed case so the preset is covered across the boundary.

Reproducer run against acb47ed

Add to LanceScannerFullTextSearchTest:

@Test
void testCodeAnalyzerBuilderCreatesIndex() throws Exception {
  ScalarIndexParams params = InvertedIndexParams.builder().analyzer("code").build();
  runFtsQuery(
      "memory://fts_java_code_analyzer",
      FullTextQuery.match("hello", "doc"),
      1L,
      Arrays.asList("helloWorld", "other_value"),
      params);
}

Run:

./mvnw -Djava.io.tmpdir=/home/agent/tmp/lance-gate-8829 -Dtest=LanceScannerFullTextSearchTest#testCodeAnalyzerBuilderCreatesIndex test

Expected: index creation succeeds and the query returns one row.

Observed: RuntimeException: Invalid user input: analyzer='code' requires base_tokenizer='code', got 'simple'.

[ddupg]

Agreed that #8779 should land first. Once it is merged, I’ll rebase this PR

[lance-gatekeeper]

Thanks. The current head is unchanged, so the finding remains pending until #8779 lands, this PR is rebased, and the JNI regression is added.