[Python][C++] Casting a list of struct array with null field results in invalid result #43838
Description
Describe the bug, including details regarding any error messages, version, and platform.
OS: MacOS Sonoma 14.6.1
Python: 3.10.14
pyarrow: 17.0.0
Snippet to reproduce:
import pyarrow as pa
fields = [('b', pa.null()),('g', pa.string())]
dtype = pa.list_(pa.struct(fields))
arr = pa.array([[{'b': None, 'g': None}, {'b': None, 'g': 'moo'}]], type=dtype)
carr = pa.chunked_array(arr)
ext_arr = carr.cast(dtype, safe=True)
print(ext_arr) # prints Invalid array but no exception
print(len(ext_arr)) # Len is 1
print(ext_arr[0]) # Raises ArrowIndexErrorComponent(s)
Python
Activity
[-]ChunkedArray cast to data type containing null results in ArrowIndexError when accessed[/-][+][Python][C++] Casting a list of struct array with null field results in invalid result[/+]jorisvandenbossche commentedon Aug 27, 2024
@Angel-212 thanks for the report!
The issue already happens with the plain (non-chunked) array as well. And the repr also shows it being invalid (which is then the reason that a getitem operation errors):
Add (failing) tests for remaining null-nested-field breakage
Work around null-typed nested field breakage in setitem/from_sequence…
apacheGH-43838: [C++] Preserve nested NullType metadata in array views
apacheGH-43838: [C++] Preserve nested NullType metadata in array views
Add xfail canary tests for upstream pyarrow null-nested bugs
Add (failing) tests for null-nested cast robustness in nested-pandas
Add safe_cast to avoid pyarrow null-typed nested cast bug
Add (failing) tests for remaining null-nested-field breakage
Work around null-typed nested field breakage in setitem/from_sequence…
Fix null nested casts (#514)