Polars py-1.42.1 - Parquet Metadata and Panic Fixes

Polars py-1.42.1 was published on June 30, 2026, with a practical patch set for parquet scans, query correctness, and runtime failure cases. The most useful change for data engineers is sampled resolve mode for multi file parquet metadata, which targets one of the dull but expensive parts of reading many files.

The full release notes and downloads are on the GitHub release page.

The release adds sampled resolve mode for multi file parquet metadata. That matters when a pipeline reads many parquet files and Polars needs enough metadata to plan the scan. Full metadata resolution can be useful, but it also creates work before the query has done anything visible.

The release notes do not give a benchmark or a new user facing flag, so it is best to treat this as a planner and scan improvement rather than a tuning knob. The important point is where it lands. It is in the path that decides how much file metadata must be inspected before execution can proceed.

There is also a path expansion change: Polars no longer blocks on path expansion. For batch ETL jobs that fan across many files, blocking before the scan starts is usually just hidden latency. This patch does not change the data model. It should make some file discovery paths less serial.

Small dtype series sums also get faster through an upcast sum kernel. This is a narrower improvement, but it is relevant for narrow tables, derived counters, and small integer columns that appear in staging data. The gain is not described in numbers, so the safe reading is targeted speedup, not a broad performance claim.

Several fixes in py-1.42.1 are about avoiding panics in normal query shapes. One fix covers scan_parquet with a filter over a fixed size binary column. That is a real data pipeline case, especially when parquet files carry binary identifiers, hashes, or encoded keys.

Another fix handles projection pushdown with select(len()) after groupby. Projection pushdown is meant to reduce unnecessary work, but this kind of optimizer path needs to preserve valid query plans. A panic here is not a cosmetic problem. It can turn an ordinary aggregation into a failed job.

The grouped max fix is also worth noting. The release corrects an incorrect sorted path that could return nan for grouped max. Aggregations are the point where many pipelines convert raw events into business tables. Wrong results in that stage are worse than a crash because they can pass through downstream systems as plausible data.

Temporal extraction for datetime columns with nulls now avoids another panic. That matters for ingestion flows where timestamps are optional or partially parsed. Null handling has to stay boring in data engineering code. This patch moves one more datetime path in that direction.

The database and Iceberg notes are mostly cleanup, but the cleanup is operationally relevant. The release closes remaining ResourceWarning leaks in database and Iceberg tests, including database test leaks called out in both enhancements and bug fixes. Test leaks are not the same as production leaks, but they are useful signals. They usually point at handles that deserve tighter lifetime management.

For lakehouse users, there is a fix for a pyiceberg key dot HDFS prefix filter. The release notes do not expand the scenario, so this should not be read as a general Iceberg compatibility rewrite. It is a specific prefix filter fix around HDFS style keys.

AWS credential handling also gets a correction for expiry time loading with DeferredRefreshableCredentials. In long running ETL workers, credential refresh bugs tend to show up after a job has already opened files, planned work, and burned time. A fix in this area is modest but useful for object store reads that depend on refreshed AWS sessions.

The remaining changes include decimal dynamic float cast behavior and safer replace handling when old or new contain Expr or object dtype values. These are not headline items, but they are the kind of type edge cases that surface in transformation code before loading into warehouse tables.

There is one clear deprecation. The strict parameter of pl.concat is deprecated and should be replaced with how='horizontal_extend'.

That is a small API migration, but it is worth searching for in shared ETL libraries before bumping the dependency. Concatenation often sits in helper code that combines extracted frames, staged chunks, or per partition results. Moving to the explicit how value should make intent clearer and avoid a later removal surprise.

The other development notes are mostly internal project hygiene. The project disallows tokio::spawn through clippy, improves Makefile virtual environment robustness, removes legacy prelude glob imports from leaf files, updates mypy to the new 2.x release, and makes pyrefly checks part of make pre-commit. Useful for contributors, but less important for users unless they build Polars from source or run the project test suite.

• Release page: GitHub release page
• Repository: project repository
• Tag: py-1.42.1