CloudQuery Snowflake Destination Skipped Later Tables in a Batch

CloudQuery is an ELT sync engine that pulls APIs into warehouses and object stores. In the last seven days the CloudQuery monorepo landed 15 commits (640 insertions, 335 deletions). The change that matters for operators is a Snowflake destination bug: schema lookup for a batch of tables only ever saw the first name, so later tables never got ALTER TABLE during migrate.

getTableInfoBatch in the Snowflake destination builds one information_schema.columns query for up to 200 names. The filter is UPPER(table_name) = ANY (SELECT COLUMN1 FROM VALUES ...). The VALUES list was assembled as a single wide row: (?,?,?). Snowflake treats that as one row with many columns. COLUMN1 is only the first bound name.

The fix switches the list to one row per name, each with a single column: (?),(?),(?). A comment in the same function states the old shape would only ever see the first name.

That matches how MigrateTables uses the result. Names missing from the lookup are treated as new and go through createTableIfNotExist. Tables that already exist skip the create path and never receive the column add. A shared schema change therefore lands on the first table in the batch and nowhere else.

getTableInfo also runs SHOW PRIMARY KEYS and SHOW UNIQUE KEYS in parallel. Those are schema wide. The VALUES bug was only on the column query. Primary key metadata for later tables was then discarded because the table pointer from column lookup was nil.

The regression test creates three tables, then migrates them again with one extra string column. After the fix, getTableInfo must return that column on every table, not just the first. A fake Snowflake driver also covers 450 names split into batches of 200, 200, and 50, plus the empty list path.

values := strings.TrimSuffix(strings.Repeat("(?),", len(tableNames)), ",")
completeSQL := sqlTableInfoStart + values + sqlTableInfoEnd

If you sync into Snowflake and a source added columns on more than one table, inspect later tables for missing columns. Recreate or alter them, then pick up destination v5.2.14.

Release Please cut plugins-destination-snowflake v5.2.14 the same day. The changelog has one bug fix line: migrating only the first table in a batch. That is the version to pin if you migrate more than one table per sync.

Two days later plugins-destination-azblob v4.5.11 shipped. The azblob changelog is dependency only: Azure SDK, Arrow v18.7.0, plugin-sdk/v4 v4.95.3, golang.org/x/net v0.56.0, and a google.golang.org/grpc v1.82.1 security bump. No write path change is described. Treat it as a routine plugin bump unless a scanner is waiting on grpc.

Most of the 15 commits in this window are the CI bot. The Snowflake VALUES change is the one operator facing code edit.

The CLI bumped plugin-pb-go to v1.27.17 in cli/go.mod and in the source plugin scaffold go.mod template. That library version lists two fixes: protobuf-go v1.36.12, and retries when a plugin download drops in the middle of the body.

The protobuf bump also landed directly on the CLI in a separate commit (v1.36.11 to v1.36.12). The protobuf-go notes mention google.protobuf.Empty serialization and a recursion limit in prototext. Nothing in this window changes plugin RPC shape from the CloudQuery side.

If cloudquery plugin install fails on flaky links, a newer CLI with plugin-pb-go v1.27.17 is the first thing to try. New source plugins from the scaffold inherit the same pin.

Each destination keeps its own module file, so these bumps do not fan out across plugins. plugins/destination/bigquery/go.mod moved cloud.google.com/go/bigquery from v1.79.1 to v1.80.0. BigQuery and GCS both took google.golang.org/api v0.293.0, a regenerated discovery client drop with no CloudQuery code edits.

aws-sdk-go-v2 moved in five destinations: S3, Firehose, PostgreSQL, MongoDB, and Gremlin. The S3 bump is in plugins/destination/s3/go.mod via this monorepo update (v1.43.4 to v1.43.6 on the core SDK, plus matching service/s3 and STS modules). The SDK notes a clock skew cache that could stick after client and server clocks realigned. There is no CloudQuery test or comment tying that to a sync failure.

A separate hackernews source change replaces archived github.com/golang/mock with go.uber.org/mock v0.6.0 and regenerates mocks. That is test plumbing. Coverage markdown for azblob and Snowflake was refreshed by the bot.

Snowflake operators should upgrade the destination plugin to v5.2.14 and check tables that were not first in a migrate batch for missing columns after a source schema change.

CLI users who pull plugins over unreliable networks should pick up plugin-pb-go v1.27.17. The rest of this window is dependency churn. Do not expect write path changes from the BigQuery, GCS, S3, or azblob bumps listed above.

The fake driver in table_info_test.go is now the right place to hang more VALUES clause cases if another dialect quirk shows up. The 200 name batch split is covered. Mixed type columns and forced migrate are not.