Decaf Blog In-depth updates on Decaf software, tooling, and open-source experiments

ingestr v1.0.39 - MongoDB CDC and Connector Expansion

3 min read

bruin-data ingestr v1.0.39 shipped on June 23, 2026 with MongoDB change data capture as the headline addition for replication style ELT workloads. The release widens the connector catalog, adds pipeline metadata columns, and fixes several load path edge cases for DuckDB and Arrow based schema evolution.

The full release notes and downloads are on the GitHub release page.

MongoDB CDC source and catalog wiring

The largest functional change is a new MongoDB CDC source landed in pull request #837. Operators can now pull document level changes from MongoDB into ingestr destinations without a separate CDC tool in front of the pipeline. The server connector catalog gained matching MongoDB CDC schemes, and the project moved those entries to a dedicated catalog slot rather than overloading a generic MongoDB connector label.

Benchmark ingestion paths for MongoDB also received optimization work in #841. That matters when you run repeated test loads or compare connector performance across environments. The catalog and runtime changes ship together in v1.0.39, so server UI users and CLI users should see the same MongoDB CDC options after upgrade.

Twilio and streaming queue broker sources

Two additional source types expand what ingestr can ingest without custom glue code.

The Twilio source covers messaging and communications API data. Catalog registration landed in the same release, and table validation now accepts granularity modifiers through isValidTable. That reduces false rejections when you model Twilio resources at different time grains.

Streaming queue broker sources arrived in #829. For teams that treat message queues as ingestion front ends, this adds a native path into ingestr rather than exporting queue payloads to files first. Pair that with the restored streaming transform coverage from #838 if your pipelines apply transforms on the streaming path rather than batch reloads.

Load metadata, DuckDB staging, and schema evolution

Pipeline observability gets a concrete column addition in #833: an ingestr load timestamp column written during ingest. That gives downstream models a stable audit field for when rows entered the warehouse, separate from source system timestamps.

DuckDB replace mode staging now respects the target schema in #842. Previously, staging table placement could drift from the schema you configured on the destination. For replace strategies that drop and reload tables, schema alignment reduces surprise permission errors and manual DDL fixes.

Schema evolution hit an Arrow ownership bug fixed in #840. Arrow buffers passed between evolution steps could fail when ownership was not transferred cleanly. If you saw intermittent failures during automatic column adds or type promotions, this release is worth testing against your worst case tables.

CLI parity, container health checks, and operator coverage

Pull request #839 brings CLI parity with the server UI. Teams that script ingestr from CI or a scheduler get the same connector and option surface the web UI exposes, which cuts down on “works in the UI, fails in cron” drift.

Container operators get a small but practical fix in #836: curl is back in the OCI image so health check workflows that probe HTTP endpoints keep working. That is an infrastructure detail, not a data model change, yet it affects anyone running ingestr as a long lived service behind a load balancer.

This is a stable release, not a prerelease or release candidate.

Where to get it