Iceberg + REST Catalog support

[derekperkins]

Dataform cannot model PROJECT.CATALOG.NAMESPACE.TABLE as a native table or incremental target. Dataform’s bigquery.iceberg configuration is also for three-part BigQuery-managed Iceberg, not the Lakehouse REST catalog. We're hoping to adopt dataform + iceberg, but where GCP is recommending the REST catalog over the old BigQuery Managed Iceberg, we don't want to start on a deprecated path

[ihistand]

The four-part naming is real, but for read-only sources there's a workaround that works today with no core change.

resolveTarget wraps the whole dotted identifier in a single pair of backticks
(core/compilation_sql/index.ts), and nothing validates dots inside the dataset part, so a
declaration can carry catalog and namespace together:

-- definitions/my_table.sqlx
config {
  type: "declaration",
  database: "my-project",
  schema: "my-catalog.my-namespace",
  name: "my-table"
}

compiles to `my-project.my-catalog.my-namespace.my-table`, which is exactly the P.C.N.T
form. Same idea in actions.yaml, where the keys are project and dataset, or via
declare({ ... }) in a JS file.

Caveats: the action's readable name is now a four-dot string, so --actions selectors and any
--schema-suffix / schema override will treat it oddly. It's fine for sources you only ref();
I wouldn't lean on it for anything else. (I read this off the compiler rather than running it
against a live catalog, so worth a quick check on your side.)

On the incremental-target half, I don't think that one is Dataform's to solve. BigQuery has
read-only access to tables through the Iceberg REST catalog endpoint, so there's no DML or CTAS
for Dataform to emit. Anything Dataform materializes has to go through an engine that can write.

It's also worth separating the two things being recommended: the REST catalog endpoint is
recommended over the older Custom Apache Iceberg catalog for BigQuery endpoint - catalog
endpoint vs catalog endpoint. BigQuery-managed Iceberg tables aren't the deprecated side of that
comparison, and they're the writable ones. So bigquery.iceberg isn't a dead path for tables you
want Dataform to build; it's the only path where BigQuery can do the writing.

[derekperkins]

Thanks for the response and the query workaround. BigQuery has supported DML in preview for the REST catalog since June. Since my primary use case for dataform is data pipelines DML, I'm hoping support can land. I'll try out your combo schema workaround to see if that works for that.

https://docs.cloud.google.com/bigquery/docs/release-notes#June_25_2026