You can run dbt using the following tools:
- In your browser with the dbt Cloud IDE
- On the command line interface using the dbt Cloud CLI or open-source dbt Core.
A key distinction with the tools mentioned, is that dbt Cloud CLI and IDE are designed to support safe parallel execution of dbt commands, leveraging dbt Cloud's infrastructure and its comprehensive features. In contrast,
dbt-core doesn't support safe parallel execution for multiple invocations in the same process. Learn more in the parallel execution section.
dbt Cloud allows for parallel execution of commands, enhancing efficiency without compromising data integrity. This enables you to run multiple commands at the same time, however it's important to understand which commands can be run in parallel and which can't.
dbt-core doesn't support safe parallel execution for multiple invocations in the same process, and requires users to manage concurrency manually to ensure data integrity and system stability.
To ensure your dbt workflows are both efficient and safe, you can run different types of dbt commands at the same time (in parallel) — for example,
dbt build (write operation) can safely run alongside
dbt parse (read operation) at the same time. However, you can't run
dbt build and
dbt run (both write operations) at the same time.
dbt commands can be
|These commands perform actions that change data or metadata in your data platform.
Limited to one invocation at any given time, which prevents any potential conflicts, such as overwriting the same table in your data platform at the same time.
|These commands involve operations that fetch or read data without making any changes to your data platform.
Can have multiple invocations in parallel and aren't limited to one invocation at any given time. This means read commands can run in parallel with other read commands and a single write command.