v0.19.0

Added

• LLMReranker now supports custom prompts as well as custom response parsers allowing for other ranking scales apart from default 1-5.
• pw.io.kafka.write and pw.io.nats.write now support ColumnReference as a topic name. When a ColumnReference is provided, each message's topic is determined by the corresponding column value.
• pw.io.python.write accepting ConnectorObserver as an alternative to pw.io.subscribe.
• pw.io.iceberg.read and pw.io.iceberg.write now support S3 as data backend and AWS Glue catalog implementations.
• All output connectors now support the sort_by field for ordering output within a single minibatch.
• A new UDF executor pw.udfs.fully_async_executor. It allows for creation of non-blocking asynchronous UDFs which results can be returned in the future processing time.
• A Future data type to represent results of fully asynchronous UDFs.
• pw.Table.await_futures method to wait for results of fully asynchronous UDFs.
• pw.io.deltalake.write now supports partition columns specification.

Changed

• BREAKING: Changed the interface of LLMReranker, the use_logit_bias, cache_strategy, retry_strategy and kwargs arguments are no longer supported.
• BREAKING: LLMReranker no longer inherits from pw.UDF
• BREAKING: pw.stdlib.utils.AsyncTransformer.output_table now returns a table with columns with Future data type.
• pw.io.deltalake.read can now read append-only tables without requiring explicit specification of primary key fields.