concurrent_batches

💡Did you know...

Available from dbt v1.9 or with the dbt Cloud "Latest" release track.

Project file

dbt_project.yml

models:
  +concurrent_batches: true

Config block

models/my_model.sql

{{
  config(
    materialized='incremental',
    concurrent_batches=true,
    incremental_strategy='microbatch'
        ...
  )
}}
select ...

Definition

concurrent_batches is an override which allows you to decide whether or not you want to run batches in parallel or sequentially (one at a time).

For more information, refer to how batch execution works.

Example

By default, dbt auto-detects whether batches can run in parallel for microbatch models. However, you can override dbt's detection by setting the concurrent_batches config to false in your dbt_project.yml or model .sql file to specify parallel or sequential execution, given you meet these conditions:

• You've configured a microbatch incremental strategy.
• You're working with cumulative metrics or any logic that depends on batch order.

Set concurrent_batches config to false to ensure batches are processed sequentially. For example:

dbt_project.yml

models:
  my_project:
    cumulative_metrics_model:
      +concurrent_batches: false

models/my_model.sql

{{
  config(
    materialized='incremental',
    incremental_strategy='microbatch'
    concurrent_batches=false
  )
}}
select ...