Skip to main content

Setting up dbt State

This page walks you through setting up dbt State across dbt v1, dbt v2, and dbt platform.

Prerequisites​

Before you set up dbt State, make sure you have:

  • A supported dbt version: dbt State is natively available in dbt platform, dbt v2, and dbt v1.13. It's also available as a plugin for dbt v1.7–1.12.
  • A supported data platform: Snowflake, Databricks, BigQuery, or Redshift. More warehouses are on the roadmap.
  • A dbt account: dbt State uses your dbt account for authentication, even if you're running dbt locally or in a non-interactive environment like CI/CD pipelines or an external orchestrator. Refer to About dbt State for sign-up details, and dbt State usage and pricing for pricing details. Note that dbt State isn't available on legacy Starter plan. Please contact dbt Labs if that applies to you.

Setting up dbt State​

dbt State requires a dbt account, even when running dbt locally, to authenticate to the dbt State service. No need to migrate your existing workflows or purchase any dbt seats.

  1. Log in to your dbt account and start your dbt State trial.

    (Applies to dbt v1.13 and later)

    1. Navigate to your project:

      cd to/your/project

    2. Log in to your dbt account:

      dbt login

      This opens a browser window where you can log in to or create a free dbt account. If you're creating a new account:

      1. Enter your name and password.
      2. Select Enable a trial of dbt State.
      3. Agree to the Terms of Service and click Continue.
      4. Sign in to dbt platform.
      5. Set up an authentication method.
      6. Verify your email address.

    3. Agree to the dbt State Terms of Service and click Start 30-day trial.

      Once started, you cannot pause the trial. After the trial ends, your account receives a free monthly DATT allowance, which varies by plan. For details, refer to dbt State trial and billing.

    You can also enable or disable dbt State per run using CLI flags: --manage-state or --no-manage-state, or set the DBT_ENGINE_MANAGE_STATE environment variable.

    To enable dbt State for everyone on your project, add manage_state: true to the flags: block in dbt_project.yml:

    flags:
      manage_state: true

  2. Run dbt run or dbt build.

    • First run: With no prior state to compare against, dbt builds every node from scratch. dbt State captures metadata from this run for future comparisons.
    • Later runs: dbt State compares each node's logic and upstream data against previous builds across all environments. It rebuilds a node when its logic has changed, or when its upstream data has changed and its last build is older than the lag_tolerance window. Otherwise, it skips or clones an existing node.

    To see how dbt State optimizes your runs, refer to dbt State usage examples.

  3. Set up your orchestration environment.

  4. Verify dbt State is active.

  5. Invite your team to use dbt State.

Verifying dbt State is active​

After a run completes, confirm that dbt State is active:

  • From the CLI: Check the log output for a message like dbt State adapter: dbt-state v2.10.1 is enabled.
  • From dbt platform: Check the logs view of built resources to see if they were built or reused, or open the Explain tab on the job run details page to see the State decision for each resource.

If dbt State isn't behaving as expected, refer to Debugging dbt State.

Inviting team members​

The more team members you have using dbt State, the better it gets; more team members means more opportunities to clone existing nodes rather than rebuilding them. To invite colleagues:

  • From the CLI (dbt v1.13+): Have your colleagues run dbt login after dbt State is enabled on the account.
  • From dbt platform: Go to Account settings > Users and click Invite users. For more information, refer to Invite new users.

Configuring lag tolerance​

Lag tolerance controls how long dbt State waits before rebuilding a node once its upstream data changes. A node rebuilds only when its last build is older than the tolerance window and its upstream data has changed. You can set it at the project, environment, or model level. If not configured, lag_tolerance defaults to 45m. We recommend starting with the following Jinja expression:

dbt_project.yml
models:
  +state:
    lag_tolerance: "{{ '4h' if target.name == 'prod' else '7d' }}"

In this example, models in the prod target rebuild once their last build is more than 4 hours old and their upstream data has changed. In all other environments, models rebuild once their last build is more than 7 days old and their upstream data has changed.

For more details, refer to the lag_tolerance config reference.

tip

To help you tune lag_tolerance values, the dbt State page on the dbt platform provides lag tolerance recommendations based on your models' 30-day build history, so you can see which models would benefit from a higher tolerance.

Debugging dbt State​

If dbt State is behaving unexpectedly, you can prepend your run command with the DBT_ENGINE_MANAGE_STATE environment variable to isolate the issue:

DBT_ENGINE_MANAGE_STATE=0 dbt run --target dev --select "customers"

To see which decision dbt State made for each node after a run and why, you can run the (Applies to dbt v2.0 and later) dbt state explain command. If you use the dbt platform, the same information is available without running a command — go to the Explain tab on the job run details page to see the full decision breakdown for each node.

Next steps​

Was this page helpful?

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

0
Loading