Critically, in order to execute dbt, environments define three variables:
- The version of dbt Core that will be used to run your project
- The warehouse connection information (including the target database/schema settings)
- The version of your code to execute
Each dbt Cloud project can have only one development environment, but there is no limit to the number of deployment environments, providing you the flexibility and customization to tailor the execution of scheduled jobs.
Use environments to customize settings for different stages of your project and streamline the execution process by using software engineering principles. This page will detail the different types of environments and how to intuitively configure your development environment in dbt Cloud.
Types of environments
In dbt Cloud, there are two types of environments:
- Deployment environment — Determines the settings used when jobs created within that environment are executed.
- Development environment — Determines the settings used in the dbt Cloud IDE or dbt Cloud CLI, for that particular project.
Each dbt Cloud project can only have a single development environment but can have any number of deployment environments.
|Development Environments||Deployment Environments|
|Determines settings for||dbt Cloud IDE or dbt Cloud CLI||dbt Cloud Job runs|
|How many can I have in my project?||1||Any number|
For users familiar with development on dbt Core, each environment is roughly analogous to an entry in your
profiles.yml file, with some additional information about your repository to ensure the proper version of code is executed. More info on dbt core environments here.
Common environment settings
Both development and deployment environments have a section called General Settings, which has some basic settings that all environments will define:
|Setting||Example Value||Definition||Accepted Values|
|Name||Production||The environment name||Any string!|
|Environment Type||Deployment||The type of environment||[Deployment, Development]|
|dbt Version||1.4 (latest)||The dbt version used||Any dbt version in the dropdown|
|Default to Custom Branch||☑️||Determines whether to use a branch other than the repository’s default||See below|
|Custom Branch||dev||Custom Branch name||See below|
- dbt Cloud allows users to select any dbt release. At this time, environments must use a dbt version greater than or equal to v1.0.0; lower versions are no longer supported.
- If you select a current version with
(latest)in the name, your environment will automatically install the latest stable version of the minor version selected.
Git repository caching
At the start of every job run, dbt Cloud clones the project's Git repository so it has the latest versions of your project's code and runs
dbt deps to install your dependencies.
For improved reliability and performance on your job runs, you can enable dbt Cloud to keep a cache of the project's Git repository. So, if there's a third-party outage that causes the cloning operation to fail, dbt Cloud will instead use the cached copy of the repo so your jobs can continue running as scheduled.
dbt Cloud caches your project's Git repo after each successful run and retains it for 8 days if there are no repo updates. It caches all packages regardless of installation method and does not fetch code outside of the job runs.
To enable Git repository caching, select Account settings from the gear menu and enable the Repository caching option.Example of the Repository caching option
This feature is only available on the dbt Cloud Enterprise plan.
Custom branch behavior
By default, all environments will use the default branch in your repository (usually the
main branch) when accessing your dbt code. This is overridable within each dbt Cloud Environment using the Default to a custom branch option. This setting have will have slightly different behavior depending on the environment type:
- Development: determines which branch in the dbt Cloud IDE or dbt Cloud CLI developers create branches from and open PRs against.
- Deployment: determines the branch is cloned during job executions for each environment.
For more info, check out this FAQ page on this topic!
Extended attributes are retrieved and applied only at runtime when
profiles.yml is requested for a specific Cloud run. Extended attributes are currently not taken into consideration for Cloud-specific features such as PrivateLink or SSH Tunneling that do not rely on
Extended Attributes is a feature that allows users to set a flexible profiles.yml snippet in their dbt Cloud Environment settings. It provides users with more control over environments (both deployment and development) and extends how dbt Cloud connects to the data platform within a given environment.
Extended Attributes is a text box extension at the environment level that overrides connection or environment credentials, including any custom environment variables. You can set any YAML attributes that a dbt adapter accepts in its
Something to note, Extended Attributes doesn't mask secret values. We recommend avoiding setting secret values to prevent visibility in the text box and logs.Extended Attributes helps users add profiles.yml attributes to dbt Cloud Environment settings using a free form text box.
If you're developing in the dbt Cloud IDE, dbt Cloud CLI, or orchestrating job runs, Extended Attributes parses through the provided YAML and extracts the
profiles.yml attributes. For each individual attribute:
If the attribute exists in another source (such as your project settings), it will replace its value (like environment-level values) in the profile. It also overrides any custom environment variables.
If the attribute doesn't exist, it will add the attribute or value pair to the profile.
Only the top-level keys are accepted in extended attributes. This means that if you want to change a specific sub-key value, you must provide the entire top-level key as a JSON block in your resulting YAML. For example, if you want to customize a particular field within a service account JSON for your BigQuery connection (like 'project_id' or 'client_email'), you need to provide an override for the entire top-level
keyfile_json main key/attribute using extended attributes. Include the sub-fields as a nested JSON block.
The following code is an example of the types of attributes you can add in the Extended Attributes text box:
Create a development environment
To create a new dbt Cloud development environment:
- Navigate to Deploy -> Environments
- Click Create Environment.
- Select Development as the environment type.
- Fill in the fields under General Settings and Development Credentials.
- Click Save to create the environment.
Set developer credentials
To use the dbt Cloud IDE or dbt Cloud CLI, each developer will need to set up personal development credentials to your warehouse connection in their Profile Settings. This allows you to set separate target information and maintain individual credentials to connect to your warehouse.Creating a development environment
Deployment environments in dbt Cloud are necessary to execute scheduled jobs and use other features. A dbt Cloud project can have multiple deployment environments, allowing for flexibility and customization. However, a dbt Cloud project can only have one deployment environment that represents the production source of truth.
To learn more about dbt Cloud deployment environments and how to configure them, refer to the Deployment environments page. For our best practices guide, read dbt Cloud environment best practices for more info.