Problem
SedonaDB currently depends on DataFusion 54.1.0, which contains a logical optimizer correctness bug in PushDownLeafProjections when nested-field access is applied to a struct produced by Unnest.
For a plan shaped like:
Projection: get_field(dump, "geom")
Unnest: dump
Projection: ST_Dump(geometry) AS dump
the optimizer can push get_field(dump, "geom") below Unnest. At that point dump is still List<Struct<...>>, rather than the post-unnest Struct<...>, and optimization fails with:
Optimizer rule 'push_down_leaf_projections' failed
caused by
Execution error: Cannot access field at argument 1: type List(Struct(...))
is not Struct, Map, or Null
This is a plan-correctness bug, not a security issue.
Upstream status
The bug was reported and fixed upstream:
The reduced reproducer fails with DataFusion 54.1.0 and succeeds on DataFusion main at e1942b144.
Temporary SedonaDB workaround
Until SedonaDB upgrades to a DataFusion release containing apache/datafusion#22620, sedona-query-planner replaces DataFusion's optimizer rule named push_down_leaf_projections with a compatibility rule that prevents leaf projection pushdown across Unnest.
Regression coverage includes both a DataFusion-only List<Struct> plan and SedonaDB's ST_Dump → unnest → geom workflow.
Action item
Problem
SedonaDB currently depends on DataFusion 54.1.0, which contains a logical optimizer correctness bug in
PushDownLeafProjectionswhen nested-field access is applied to a struct produced byUnnest.For a plan shaped like:
the optimizer can push
get_field(dump, "geom")belowUnnest. At that pointdumpis stillList<Struct<...>>, rather than the post-unnestStruct<...>, and optimization fails with:This is a plan-correctness bug, not a security issue.
Upstream status
The bug was reported and fixed upstream:
PushDownLeafProjectionscrossingUnnest, including the same-named pre/post-unnest column problem.Unnestas a semantic barrier for leaf-projection pushdown.LogicalPlan::Unnestexpression/rebuild contracts consistent while retaining the semantic barrier.The reduced reproducer fails with DataFusion 54.1.0 and succeeds on DataFusion
mainate1942b144.Temporary SedonaDB workaround
Until SedonaDB upgrades to a DataFusion release containing apache/datafusion#22620,
sedona-query-plannerreplaces DataFusion's optimizer rule namedpush_down_leaf_projectionswith a compatibility rule that prevents leaf projection pushdown acrossUnnest.Regression coverage includes both a DataFusion-only
List<Struct>plan and SedonaDB'sST_Dump → unnest → geomworkflow.Action item
PushDownLeafProjectionscompatibility rule and its registration, while retaining appropriate regression coverage against the upstream rule.