Polars DataFrames

Install pipewhy[polars] to capture local eager Polars DataFrames. The initial adapter supports explicitly declared Int64 and String columns, including nulls when the field permits them. Other types and LazyFrame inputs are rejected.

from pathlib import Path
import polars as pl
from pipewhy.adapters.polars import PolarsSource
from pipewhy.cli.capture import scan_source
from pipewhy.formats import PolarsField, PolarsLimits, PolarsSchema, polars_source_version
from pipewhy.snapshots import LocalSnapshotStore

frame = pl.DataFrame({
    "id": pl.Series([1, 2], dtype=pl.Int64),
    "name": pl.Series(["Ada", None], dtype=pl.String),
})
schema = PolarsSchema((PolarsField(0, "id", "int64"), PolarsField(1, "name", "string", True)))
limits = PolarsLimits(1000, 10, 1024, 1000000, 100, 10000)
source = PolarsSource("people", frame, schema, limits)
version = polars_source_version(frame, schema=schema, limits=limits)
snapshot = scan_source(source, LocalSnapshotStore(Path("snapshots")), version)

The source version includes the schema and ordered records. Capture stages the records and checks the DataFrame again before publishing. Callers must prevent concurrent mutation: the check detects accidental drift, but is not an atomic transaction. The staged-byte limit bounds encoded records and offsets, not the memory already occupied by the DataFrame. Strict normalization is the default; logical normalization is opt-in. Snapshots reopen without the original frame.