Pyarrow wrapper

Module for building dags using pyarrow.

ArrowDagWrapper dataclass

Helper call for adding pyarrow Nodes to a Dag.

filter_stream(predicate, stream, *args, **kwargs)

Filter a stream Node of type pa.Table.

get_column(stream, key)

Return a column from a stream node of type pa.Table.

latest_by_keys(stream, keys)

Build a state of the latest row by keys.

source_table(schema, name=None)

Add a source stream of type pa.Table.

table_stream(function, schema)

Add a stream node of output type pa.Table