CHANGELOG.md

Changelog

53.3.0 (2024-11-17)

Full Changelog

Implemented enhancements:

• PartialEq of GenericByteViewArray (StringViewArray / ByteViewArray) that compares on equality rather than logical value #6679 [arrow]
• Need a mechanism to handle schema changes due to dictionary hydration in FlightSQL server implementations #6672 [arrow] [arrow-flight]
• Support encoding Utf8View columns to JSON #6642 [arrow]
• Implement append_n for BooleanBuilder #6634 [arrow]
• Some take optimizations #6621 [arrow]
• Error Instead of Panic On Attempting to Write More Than 32769 Row Groups #6591 [parquet]
• Make casting from a timestamp without timezone to a timestamp with timezone configurable #6555
• Add record_batch! macro for easy record batch creation #6553 [arrow]
• Support Binary --> Utf8View casting #6531 [arrow]
• downcast_primitive_array and downcast_dictionary_array are not hygienic wrt imports #6400 [arrow]
• Implement interleave_record_batch #6731 [arrow] (waynexia)
• feat: record_batch! macro #6588 [arrow] (ByteBaker)

Fixed bugs:

• Signed decimal e-notation parsing bug #6728 [arrow]
• Add support for Utf8View -> numeric in can_cast_types #6715
• IPC file writer produces incorrect footer when not preserving dict ID #6710 [arrow]
• parquet from_thrift_helper incorrectly checks index #6693 [parquet]
• Primitive REPEATED fields not contained in LIST annotated groups aren't read as lists by record reader #6648 [parquet]
• DictionaryHandling does not recurse into Map fields #6644 [arrow] [arrow-flight]
• Array writer output empty when no record is written #6613 [arrow]
• Archery Integration Test with c# failing on main #6577 [arrow]
• Potential unsoundness in filter_run_end_array #6569 [arrow]
• Parquet reader can generate incorrect validity buffer information for nested structures #6510 [parquet]
• arrow-array ffi: FFI_ArrowArray.null_count is always interpreted as unsigned and initialized during conversion from C to Rust. #6497 [arrow]

Documentation updates:

• Minor: Document pattern for accessing views in StringView #6673 [arrow] (alamb)
• Improve Array::is_nullable documentation #6615 [arrow] (findepi)
• Minor: improve docs for ByteViewArray->ByteArray From impl #6610 [arrow] (alamb)

Performance improvements:

• Speed up filter_run_end_array #6712 [arrow] (Dandandan)

Closed issues:

• Incorrect like results for pattern starting/ending with % percent and containing escape characters #6702 [arrow]

Merged pull requests:

• Fix signed decimal e-notation parsing #6729 [arrow] (gruuya)
• Clean up some arrow-flight tests and duplicated code #6725 [arrow] [arrow-flight] (itsjunetime)
• Update PR template section about API breaking changes #6723 (findepi)
• Support for casting StringViewArray to DecimalArray #6720 [arrow] (tlm365)
• File writer preserve dict bug #6711 [arrow] (brancz)
• Add filter_kernel benchmark for run array #6706 [arrow] (delamarch3)
• Fix string view ILIKE checks with NULL values #6705 [arrow] (findepi)
• Implement logical_null_count for more array types #6704 [arrow] (findepi)
• Fix LIKE with escapes #6703 [arrow] (findepi)
• Speed up filter_bytes #6699 [arrow] (Dandandan)
• Minor: fix misleading comment in byte view #6695 [arrow] (jayzhan211)
• minor fix on checking index #6694 [parquet] (jp0317)
• Undo run end filter performance regression #6691 [arrow] (delamarch3)
• Reimplement PartialEq of GenericByteViewArray compares by logical value #6689 [arrow] (tlm365)
• feat: expose known_schema from FlightDataEncoder #6688 [arrow] [arrow-flight] (nathanielc)
• Update hashbrown requirement from 0.14.2 to 0.15.1 #6684 [parquet] [arrow] (dependabot[bot])
• Support Duration in JSON Reader #6683 [arrow] (simonvandel)
• Check predicate and values are the same length for run end array filter safety #6675 [arrow] (delamarch3)
• [ffi] Fix arrow-array null_count error during conversion from C to Rust #6674 [arrow] (adbmal)
• Support Utf8View for bit_length kernel #6671 [arrow] (austin362667)
• Fix string view LIKE checks with NULL values #6662 [arrow] (findepi)
• Improve documentation for nullif kernel #6658 [arrow] (alamb)
• Improve test_auth error message when contains() fails #6657 [arrow] [arrow-flight] (findepi)
• Let std::fmt::Debug for StructArray output Null/Validity info #6655 [arrow] (XinyuZeng)
• Include offending line number when processing CSV file fails #6653 [arrow] (findepi)
• feat: add write_bytes for GenericBinaryBuilder #6652 [arrow] (tisonkun)
• feat: Support Utf8View in JSON serialization #6651 [arrow] (jonmmease)
• fix: include chrono-tz in flight sql cli #6650 [arrow] [arrow-flight] (crepererum)
• Handle primitive REPEATED field not contained in LIST annotated group #6649 [parquet] (zeevm)
• Implement append_n for BooleanBuilder #6646 [arrow] (delamarch3)
• fix: recurse into Map datatype when hydrating dictionaries #6645 [arrow] [arrow-flight] (nathanielc)
• fix: enable TLS roots for flight CLI client #6640 [arrow] [arrow-flight] (crepererum)
• doc: Clarify take kernel semantics #6632 [arrow] (viirya)
• Return error rather than panic when too many row groups are written #6629 [parquet] (etseidl)
• Fix test feature selection so all feature combinations work as expected #6626 [parquet] (itsjunetime)
• Add Parquet RowSelection benchmark #6623 [parquet] (XiangpengHao)
• Optimize take_bits to optimize take_boolean / take_primitive / take_byte_view: up to -25% #6622 [arrow] (Dandandan)
• Make downcast macros hygenic (#6400) #6620 [arrow] (tustvold)
• Update proc-macro2 requirement from =1.0.88 to =1.0.89 #6618 [arrow] [arrow-flight] (dependabot[bot])
• Fix arrow-json writer empty #6614 [arrow] (gwik)
• Add ParquetObjectReader::with_runtime #6612 [parquet] [arrow] (itsjunetime)
• Re-enable C# arrow flight integration test #6611 [arrow] (alamb)
• Add Array::logical_null_count for inspecting number of null values #6608 [parquet] [arrow] (findepi)
• Added casting from Binary/LargeBinary to Utf8View #6592 [arrow] (ngli-me)
• Parquet AsyncReader: Don't panic when empty offset_index is Some([]) #6582 [parquet] (jroddev)
• Skip writing down null buffers for non-nullable primitive arrays #6524 [parquet] (bkirwi)

* This Changelog was automatically generated by github_changelog_generator