map: no in-place append — merge + to_owned makes an array accumulated over a loop O(n²) #374

Summary

There is no way to append to an array in place. The idiom is a map mapping {"path": "data.log", "logic": {"merge": [{"var": "data.log"}, [ … ]]}}, and that is O(n) twice over per append: merge builds the whole new array in the arena, then the mapping result is deep-cloned with to_owned() (dataflow-rs 3.14.0 functions/map.rs:366) and the old array dropped. An array accumulated over a 1000-sweep loop is O(n²).

The same shape shows up inside a single evaluation: reduce with merge(accumulator, [x]) copies the accumulator every step, and merge is charged one op, so ops_budget does not see the quadratic cost.

Workarounds today

Batching (append to a small buffer, fold into the log every K sweeps) cuts the merge cost by K but not the O(n) arena copy the log costs every step (#371).

Proposal

A write mode on map mappings: {"path": "data.log", "append": <logic>} (or "mode": "append"), pushing the evaluated value (or extending with an array) onto the existing array in place, with no to_owned of the whole.