Skip to content

[C++][Python] pyarrow.compute.indices_nonzero segfaults on a ChunkedArray with zero chunks #51018

Description

@prathapsridharan

Describe the bug, including details regarding any error messages, version, and platform.

pyarrow.compute.indices_nonzero() segfaults when given a ChunkedArray that has zero chunks.

This is not the same thing as an empty array. A zero-length Array works, and a ChunkedArray
holding one empty chunk works. It is specifically the chunkless ChunkedArray that crashes —
which is exactly the shape you get from ordinary operations like filtering a table down to no rows,
or slicing zero rows out of it. So this is easy to hit from normal code and hard to reproduce if you
construct "an empty array" the obvious way.

The process dies with SIGSEGV; there is no Python-level exception to catch.

Minimal reproducer

import pyarrow as pa
import pyarrow.compute as pc

t = pa.table({"x": pa.array([1, 2, 3])})
empty = t.filter(pc.equal(t["x"], 99))   # matches nothing
mask = pc.equal(empty.column("x"), 1)    # ChunkedArray, num_chunks == 0, len == 0

pc.indices_nonzero(mask)                 # <- segfault

What works vs. what crashes

input num_chunks result
pa.array([], type=pa.bool_()) n/a (Array) ✅ returns []
pa.chunked_array([pa.array([], type=pa.bool_())]) 1 ✅ returns []
pa.chunked_array([], type=pa.bool_()) 0 🔴 SIGSEGV

Every one of these routes to a zero-chunk boolean ChunkedArray crashes:

pa.chunked_array([], type=pa.bool_())                        # constructed directly
pc.equal(pa.chunked_array([[1]]).filter(pa.array([False])), 1)
pc.equal(t.filter(pc.equal(t["x"], 99)).column("x"), 1)      # table filtered to no rows
pc.equal(t.slice(0, 0).column("x"), 1)                       # zero-row slice
pc.equal(pc.list_flatten(empty_list_column), 1)              # flatten of an empty list column

Expected behaviour

Return an empty index array, consistent with the zero-length and one-empty-chunk cases.

Affected versions

Reproduced on macOS (arm64), CPython 3.12, on every version I tried:

pyarrow result
18.1.0 segfault
21.0.0 segfault
24.0.0 segfault
25.0.0 segfault
25.0.1 (latest release) segfault

So it is long-standing rather than a recent regression, and there is no released version to upgrade to.

Scope

It appears specific to this kernel rather than general to zero-chunk handling. On the identical
input, unique, value_counts, sort_indices, dictionary_encode, fill_null_forward and rank
all return cleanly; list_flatten, cumulative_sum and index_in raise ordinary Python exceptions.
indices_nonzero was the only one that crashed the process.

Possibly related

Zero-chunk ChunkedArray has been a crash source before and was fixed for Slice:

This looks like the same class of defect in a different kernel.

Component(s)

C++, Python

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions