dbt - Documentation

Managing environments

What are environments?

In software engineering, environments are used to enable engineers to develop and test code without impacting the users of their software.

“Production” (or prod) refers to the environment that end users interact with, while “development” (or dev) is the environment that engineers write code in. This means that engineers can work iteratively when writing and testing new code in development, and once they are confident in these changes, deploy their code to production.

In traditional software engineering, different environments often use completely separate architecture. For example, the dev and prod versions of a website may use different servers and databases.

Data warehouses can also be designed to have separate environments – the production environment refers to the relations (i.e. schemas, tables, and views) that your end users query (often through a BI tool).

How do I maintain different environments with dbt?

dbt makes it easy to maintain separate production and development environments through the use of target - target - A set of connection details for a data warehouse (e.g. username and password), and a default schema for dbt to build relations (e.g. tables, views) in. s within a profile. A typical profile when using dbt locally will have a target named dev, and have this set as the default. This means that while making changes, your objects will be built in your development target, without affecting production queries made by your end users. Once you are confident in your changes, you can deploy the code to production, by running your dbt project with a prod target.

Targets offer the flexibility to decide how to implement your separate environments – whether you want to use separate schemas, databases, or entirely different clusters altogether! We recommend using different schemas within one data warehouse to separate your environments. This is the easiest to set up, and is the most cost effective solution in a modern cloud-based data stack.

In practice, this means that most of the details in a target will consistent across all targets, except for the schema and user credentials. If you have multiple dbt users writing code, it often makes sense for each user to have their own development environment. A pattern we've found really useful is to set your dev target schema to be dbt_<username>. User credentials should also differ across targets so that each dbt user is using their own data warehouse user.