bug: filtering a Float16 column with a numeric literal always fails

[LuciferYang]

What happens

A Float16 scalar column can be written and scanned, but any filter comparing it against a numeric literal fails while the filter is being resolved:

Invalid user input: Error resolving filter expression value < 0.0: Invalid user input: Received literal Float64(0) and could not convert to literal of type 'Float16', rust/lance-datafusion/src/logical_expr.rs:24:88, rust/lance-datafusion/src/planner.rs:949:13

Both spellings of the literal fail, value < 0.0 and value < 0. The same predicates work on a Float32 column. IS NULL and IS NOT NULL work because they carry no literal.

Reproduction

use std::sync::Arc;

use arrow_array::{Float16Array, Int32Array, RecordBatch, RecordBatchIterator};
use arrow_schema::{DataType, Field, Schema};
use half::f16;
use lance::Dataset;

#[tokio::test]
async fn float16_scalar_column() {
    let schema = Arc::new(Schema::new(vec![
        Field::new("id", DataType::Int32, false),
        Field::new("value", DataType::Float16, true),
    ]));
    let batch = RecordBatch::try_new(
        schema.clone(),
        vec![
            Arc::new(Int32Array::from_iter_values(0..3)),
            Arc::new(Float16Array::from(vec![
                f16::ZERO,
                f16::ONE,
                f16::from_f32(-1.0),
            ])),
        ],
    )
    .unwrap();
    let reader = RecordBatchIterator::new(vec![Ok(batch)], schema);
    let ds = Dataset::write(reader, "memory://f16", None).await.unwrap();

    // Fine.
    ds.scan().try_into_batch().await.unwrap();

    // Fails.
    let mut scanner = ds.scan();
    scanner.filter("value < 0.0").unwrap();
    scanner.try_into_batch().await.unwrap();
}

Cause

safe_coerce_scalar in rust/lance-datafusion/src/expr.rs has no Float16 arm in either direction. The ScalarValue::Float32 and ScalarValue::Float64 source arms accept only DataType::Float32 and DataType::Float64 as targets and return None for anything else, and there is no ScalarValue::Float16 source arm at all. logical_expr.rs:24 turns that None into the error above.

Both directions matter, and the second one is easy to miss because it does not produce an error. Adding the DataType::Float16 targets alone makes filters work, but the scalar index is then still never used: maybe_scalar in rust/lance-index/src/scalar/expression.rs calls safe_coerce_scalar on a literal the planner has already coerced to Float16, so without a ScalarValue::Float16 source arm the predicate silently becomes a refine filter. The rows come back correct, so only the plan shows it. With a BTree index on the column and 5000 rows, value = 1.0 plans as refine_filter=value = Float16(1) where a Float32 column of the same shape plans as ScalarIndexQuery: query=[value = 1]@value_idx(BTree).

Expected

A numeric literal should coerce to Float16 and filter, the way it already does for Float32 and Float64, and an indexed Float16 column should use its index.

One design decision the fix has to make: f16 overflows at 65520, so a literal like 100000 cannot be represented. Saturating it to infinity the way the Float64 to Float32 arm does answers value < 100000 correctly but makes value = 100000 match rows that really hold infinity, and this function cannot see the operator it is coercing for.

Also missing, separately

IndexType::BloomFilter rejects Float16 outright (rust/lance-index/src/scalar/bloomfilter.rs: Bloom filter index does not support data type: Float16). BTree, Bitmap and ZoneMap all accept it. That is a gap in that index rather than in literal coercion.

Found while working on #5868: the signed-zero rewrite could not be covered for Float16 because no numeric literal reaches a Float16 column in the first place.