Skip to main content

About profiles.yml

If you're using dbt from the command line, you need a profiles.yml file that contains the connection details for your data platform.

dbt platform accounts

dbt platform projects don't require a profiles.yml file unless you're developing from your local machine instead of the cloud-based UI.

About profiles.yml​

The profiles.yml file stores database connection credentials and configuration for dbt projects, including:

  • Connection details — Account identifiers, hosts, ports, and authentication credentials.
  • Target definitions — Define different environments (dev, staging, prod) within a single profile.
  • Default target — Set which environment to use by default.
  • Execution parameters — Thread count, timeouts, and retry settings.
  • Credential separation — Keep sensitive information out of version control.

The profile field in dbt_project.yml references a profile name defined in profiles.yml.

Location of profiles.yml​

Only one profiles.yml file is required and it can manage multiple projects and connections.

Fusion searches for the parent directory of profiles.yml in the following order and uses the first location it finds:

  1. --profiles-dir flag — Override for CI/CD or testing.
  2. Project root directory — Project-specific credentials.
  3. ~/.dbt/ directory (Recommended location) — Shared across all projects.

~/.dbt/profiles.yml is the recommended location for the following reasons:

  • Security — Keeps credentials out of project directories and version control.
  • Reusability — A single file for all dbt projects on the machine.
  • Separation — Connection details don't travel with project code.

When should I use project root?​

Place your profiles.yml file in the project root directory for:

  • Self-contained demo or tutorial projects.
  • Docker containers with baked-in credentials.
  • CI/CD pipelines with environment-specific configs.

Create and configure the profiles.yml file​

The easiest way to create and configure a profiles.yml file is to execute dbt init after you've installed dbt on your machine. This takes you through the process of configuring an adapter and places the file into the recommended ~/.dbt/ location.

If your project has an existing profiles.yml file, running dbt init will prompt you to amend or overwrite it. If you select the existing adapter for configuration, dbt will automatically populate the existing values.

You can also manually create the file and add it to the proper location. To configure an adapter manually, copy and paste the fields from the adapter setup instructions for dbt Core or Fusion along with the appropriate values for each.

Example configuration​

To set up your profile, copy the correct sample profile for your warehouse into your profiles.yml file and update the details as follows:

  • Profile name: Replace the name of the profile with a sensible name – it’s often a good idea to use the name of your organization. Make sure that this is the same name as the profile indicated in your dbt_project.yml file.
  • target: This is the default target your dbt project will use. It must be one of the targets you define in your profile. Commonly it is set to dev.
  • Populating your outputs:
    • type: The type of data warehouse you are connecting to
    • Warehouse credentials: Get these from your database administrator if you don’t already have them. Remember that user credentials are very sensitive information that should not be shared. May include fields like account, username, and password.
    • schema: The default schema that dbt will build objects in.
    • threads: The number of threads the dbt project will run on.

The following example highlighs the format of the profiles.yml file. Note that many of the configs are adapter-specific and their syntax varies.

~/.dbt/profiles.yml
my_project_profile:  # Profile name (matches dbt_project.yml)
  target: dev  # Default target to use
  outputs:
    dev: # Development environment
      type: adapter_type # Required: snowflake, bigquery, databricks, redshift, postgres, etc
      # Connection identifiers (placeholder examples, see adapter-specific pages for supported configs)
      account: abc123  
      database: docs_team 
      schema: dev_schema       
      # Authentication (adapter-specific)
      auth_method: username_password  
      username: username
      password_credentials: password
      # Execution settings (common across adapters)
      threads: 4   # Number of parallel threads

# Multiple profiles (for multiple projects)
my_second_project_profile:
  target: dev
  outputs:
    dev:
      type: snowflake  # Example adapter
      account: account
      user: user
      password: password
      database: database
      schema: schema
      warehouse: warehouse
      threads: 4

Environment variables​

Use environment variables to keep sensitive credentials out of your profiles.yml file. Check out the env_var reference for more information.

Example:

~/.dbt/profiles.yml
my_profile:
  target: dev
  outputs:
    dev:
      type: ADAPTER_NAME
      account: "{{ env_var("ADAPTER_ACCOUNT") }}"
      user: "{{ env_var("ADAPTER_USER") }}"
      password: "{{ env_var("ADAPTER_PASSWORD") }}"
      database: "{{ env_var("ADAPTER_DATABASE") }}"
      schema: "{{ env_var("ADAPTER_SCHEMA") }}"
      warehouse: "{{ env_var("ADAPTER_WAREHOUSE") }}"
      role: "{{ env_var("ADAPTER_ROLE") }}"
      threads: 4

User config​

You can set default values of global configs for all projects that you run using your local machine. Refer to About global configs for details.

Understanding targets in profiles​

dbt supports multiple targets within one profile to encourage the use of separate development and production environments as discussed in dbt environments.

A typical profile for an analyst using dbt locally will have a target named dev, and have this set as the default.

You may also have a prod target within your profile, which creates the objects in your production schema. However, since it's often desirable to perform production runs on a schedule, we recommend deploying your dbt project to a separate machine other than your local machine. Most dbt users only have a dev target in their profile on their local machine.

If you do have multiple targets in your profile, and want to use a target other than the default, you can do this using the --target flag when running a dbt command.

For example, to run against your prod target instead of the default dev target:

dbt run --target prod

You can use the --target flag with any dbt command, such as:

dbt build --target prod
dbt test --target dev
dbt compile --target qa

Overriding profiles and targets​

When running dbt commands, you can specify which profile and target to use from the CLI using the --profile and --target flags. These flags override what’s defined in your dbt_project.yml as long as the specified profile and target are already defined in your profiles.yml file.

To run your dbt project with a different profile or target than the default, you can do so using the followingCLI flags:

  • --profile flag — Overrides the profile set in dbt_project.yml by pointing to another profile defined in profiles.yml.
  • --target flag — Specifies the target within that profile to use (as defined in profiles.yml).

These flags help when you're working with multiple profiles and targets and want to override defaults without changing your files.

dbt run --profile my-profile-name --target dev

In this example, the dbt run command will use the my-profile-name profile and the dev target.

Understanding warehouse credentials​

We recommend that each dbt user has their own set of database credentials, including a separate user for production runs of dbt – this helps debug rogue queries, simplifies ownerships of schemas, and improves security.

To ensure the user credentials you use in your target allow dbt to run, you will need to ensure the user has appropriate privileges. While the exact privileges needed varies between data warehouses, at a minimum your user must be able to:

  • Read source data
  • Create schemas¹
  • Read system tables
Running dbt without create schema privileges

If your user is unable to be granted the privilege to create schemas, your dbt runs should instead target an existing schema that your user has permission to create relations within.

Understanding target schemas​

The target schema represents the default schema that dbt will build objects into, and is often used as the differentiator between separate environments within a warehouse.

Schemas in BigQuery

dbt uses the term "schema" in a target across all supported warehouses for consistency. Note that in the case of BigQuery, a schema is actually a dataset.

The schema used for production should be named in a way that makes it clear that it is ready for end-users to use for analysis – we often name this analytics.

In development, a pattern we’ve found to work well is to name the schema in your dev target dbt_<username>. Suffixing your name to the schema enables multiple users to develop in dbt, since each user will have their own separate schema for development, so that users will not build over the top of each other, and ensuring that object ownership and permissions are consistent across an entire schema.

Note that there’s no need to create your target schema beforehand – dbt will check if the schema already exists when it runs, and create it if it doesn’t.

While the target schema represents the default schema that dbt will use, it may make sense to split your models into separate schemas, which can be done by using custom schemas.

Understanding threads​

When dbt runs, it creates a directed acyclic graph (DAG) of links between models. The number of threads represents the maximum number of paths through the graph dbt may work on at once – increasing the number of threads can minimize the run time of your project. The default value for threads in user profiles is 4 threads.

For more information, check out using threads.

Was this page helpful?

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

0
Loading