Apache Airflow 3.3.0 - Partitioned Assets and State Store

Apache Airflow 3.3.0 was published on July 6, 2026, with asset partitioning as the main scheduler change for data teams running partitioned Dag graphs. This release is not marked as a prerelease, and the practical shift is that one upstream asset event can now drive controlled downstream partition runs through explicit mapper and wait policy behavior.

The full release notes and downloads are on the GitHub release page.

Airflow 3.3.0 expands partitioned assets beyond the base model introduced in 3.2.0. New mappers include RollupMapper, FanOutMapper, and FixedKeyMapper with SegmentWindow, covering many to one rollups, one to many fan out, and categorical windows. Time windows can use day, week, month, quarter, or year boundaries.

The important operator detail is that fan out is bounded. The new [scheduler] partition_mapper_max_downstream_keys setting caps total downstream keys per upstream event, and mapper level config can tighten that scope. For high volume asset graphs, partition routing is now more expressive without leaving the scheduler with an unbounded expansion problem.

The release also adds PartitionedAtRuntime. A Dag can declare that partition keys are assigned when the run starts instead of deriving them from an upstream asset event. That matters for pipelines where the actual partition is only known once an extraction task reads a manifest, API cursor, or table range.

Airflow 3.3.0 introduces a first class task and asset state store. Tasks can persist key value state through task_state_store, and assets can carry state through asset_state_store. Both accessors are available from the Task SDK.

The default backend is the metadata database, with a custom worker side backend configured through [workers] state_store_backend. The release notes also call out per key retention, periodic garbage collection, and optional clear_on_success, with management through the Core API and Execution API.

This is useful for retries and reruns where a task needs to remember progress without encoding that state into XComs or another side table. The tradeoff is that state now becomes part of operational hygiene. Retention policy and backend choice need the same review as Dag run history and metadata database growth.

Retry behavior is no longer limited to a fixed retries count. Airflow 3.3.0 adds pluggable retry policies, so a task can decide whether and when to retry based on custom logic, including exception specific rules or custom backoff.

The language task SDK is a larger execution change. The new Coordinator layer lets a Dag stay in Python while a task implementation runs outside Python. A task is declared with @task.stub(queue=...); the worker routes it to JavaCoordinator for JVM workloads or ExecutableCoordinator for standalone native binaries such as Go.

For mixed runtime teams, this is a cleaner boundary than shelling out from Python task code. The release notes are clear that the Coordinator layer and Java or Go SDKs are experimental in 3.3.0, so operators should treat this as an evaluation path rather than a stable contract.

The new rerun_with_latest_version setting controls whether cleared, rerun, or backfilled Dag runs use the latest bundle version or the original version from the initial run. The default is resolved from an explicit request parameter or CLI flag, then the Dag level setting, then [core] rerun_with_latest_version. If nothing is set, clear and rerun default to False, while backfills default to True.

Provider example Dags now live in dedicated bundles. Provider distributions use bundle names like apache-airflow-providers-<distribution>-example-dags, while third party providers use names like <distribution>-example-dags. REST API clients that filtered bundle_name by "dags-folder" for provider example Dags need to update that assumption.

Remote task log handler resolution has moved to airflow_shared.logging.factory. The new order checks a custom [logging] logging_config_class, then provider dispatch from [logging] remote_base_log_folder, then a legacy airflow_local_settings.py fallback that is planned for removal in Airflow 4.0. Provider remote log classes should implement a no argument from_config() method that reads airflow.providers.common.compat.sdk.conf.

Two metric changes are also worth checking in dashboards. OpenTelemetry timer and timing metrics are now Histograms instead of Gauges. The dag_processing.last_run.seconds_ago metric now carries file_path, bundle_name, and file_name tags, while the legacy name remains enabled by default and can be disabled through [metrics] legacy_names_on.

• Release page: GitHub release page
• Repository: apache/airflow
• Tag: 3.3.0