# Quickstart for dbt Core from a manual install

[Back to guides](../guides.md)

dbt Core

Quickstart

Beginner

[Menu ]()



## Introduction

When you use dbt Core to work with dbt, you will be editing files locally using a code editor, and running projects using a command line interface (CLI).

If you want to edit files and run projects using the web-based dbt Integrated Development Environment (Studio IDE), refer to the [dbt quickstarts](../guides.md). You can also develop and run dbt commands using the [dbt CLI](../docs/platform/dbt-cli-installation.md) — a dbt powered command line.

### Prerequisites

* To use dbt Core, it's important that you know some basics of the Terminal. In particular, you should understand `cd`, `ls` and `pwd` to navigate through the directory structure of your computer easily.
* Install dbt Core using the [installation instructions](../docs/local/install-dbt.md) for your operating system.
* Complete appropriate Setting up and Loading data steps in the Quickstart for dbt series. For example, for BigQuery, complete [Setting up (in BigQuery)](./bigquery.md?step=2) and [Loading data (BigQuery)](./bigquery.md?step=3).
* [Create a GitHub account](https://github.com/join) if you don't already have one.

### Create a starter project

After setting up BigQuery to work with dbt, you are ready to create a starter project with example models, before building your own models.

## Create a repository

The following steps use [GitHub](https://github.com/) as the Git provider for this guide, but you can use any Git provider. You should have already [created a GitHub account](https://github.com/join).

1. [Create a new GitHub repository](https://github.com/new) named `dbt-tutorial`.

2. Select one of the following (You can always change this setting later):

   * **Private (recommended):** To secure your environment and prevent private information (like credentials) from being public.
   * **Public:** If you need to easily collaborate and share with others, especially outside of your organization.

3. Leave the default values for all other settings.

4. Click **Create repository**.

5. Save the commands from "…or create a new repository on the command line" to use later in [Commit your changes](./manual-install.md?step=6).

## Create a project

Learn how to use a series of commands using the command line of the Terminal to create your project. dbt Core includes an `init` command that helps scaffold a dbt project.

To create your dbt project:

1. Make sure you have dbt Core installed and check the version using the `dbt --version` command:

```shell
dbt --version
```

2. Initiate the `jaffle_shop` project using the `init` command:

```shell
dbt init jaffle_shop
```

3. Navigate into your project's directory:

```shell
cd jaffle_shop
```

4. Use `pwd` to confirm that you are in the right spot:

```shell
$ pwd
> Users/BBaggins/dbt-tutorial/jaffle_shop
```

5. Use a code editor like Atom or VSCode to open the project directory you created in the previous steps, which we named jaffle\_shop. The content includes folders and `.sql` and `.yml` files generated by the `init` command.

[![The starter project in a code editor](/img/starter-project-dbt-cli.png?v=2 "The starter project in a code editor")](#)The starter project in a code editor

6. dbt provides the following values in the `dbt_project.yml` file:

dbt\_project.yml

```yaml
name: jaffle_shop # Change from the default, `my_new_project`

...

profile: jaffle_shop # Change from the default profile name, `default`

...

models:
    jaffle_shop: # Change from `my_new_project` to match the previous value for `name:`
    ...
```

## Connect to BigQuery

When developing locally, dbt connects to your data warehouse using a [profile](../docs/local/profiles.yml.md), which is a YAML file with all the connection details to your warehouse.

1. Create a file in the `~/.dbt/` directory named `profiles.yml`.
2. Move your BigQuery keyfile into this directory.
3. Copy the following and paste into the new profiles.yml file. Make sure you update the values where noted.

profiles.yml

```yaml
jaffle_shop: # this needs to match the profile in your dbt_project.yml file
    target: dev
    outputs:
        dev:
            type: bigquery
            method: service-account
            keyfile: /Users/BBaggins/.dbt/dbt-tutorial-project-331118.json # replace this with the full path to your keyfile
            project: grand-highway-265418 # Replace this with your project id
            dataset: dbt_bbagins # Replace this with dbt_your_name, e.g. dbt_bilbo
            threads: 1
            timeout_seconds: 300
            location: US
            priority: interactive
```

4. Run the `debug` command from your project to confirm that you can successfully connect:

```shell
$ dbt debug
> Connection test: OK connection ok
```

[![A successful dbt debug command](/img/successful-dbt-debug.png?v=2 "A successful dbt debug command")](#)A successful dbt debug command

### FAQs

My data team uses a different data warehouse. What should my profiles.yml file look like for my warehouse?

The structure of a profile looks different on each warehouse. Check out the [Supported Data Platforms](../docs/supported-data-platforms.md) page, and navigate to the `Profile Setup` section for your warehouse.

Why are profiles stored outside of my project?

Profiles are stored separately to dbt projects to avoid checking credentials into version control. Database credentials are extremely sensitive information and should **never be checked into version control**.

What should I name my profile?

We typically use a company name for a profile name, and then use targets to differentiate between `dev` and `prod`. Check out the docs on [environments in dbt Core](../docs/local/dbt-core-environments.md) for more information.

What should I name my target?

We typically use targets to differentiate between development and production runs of dbt, naming the targets `dev` and `prod`, respectively. Check out the docs on [managing environments in dbt Core](../docs/local/dbt-core-environments.md) for more information.

Can I use environment variables in my profile?

Yes! Check out the docs on [environment variables](../reference/dbt-jinja-functions/env_var.md) for more information.

## Perform your first dbt run

Our sample project has some example models in it. We're going to check that we can run them to confirm everything is in order.

1. Enter the `run` command to build example models:

```shell
dbt run
```

You should have an output that looks like this:

[![A successful dbt run command](/img/successful-dbt-run.png?v=2 "A successful dbt run command")](#)A successful dbt run command

## Commit your changes

Commit your changes so that the repository contains the latest code.

1. Link the GitHub repository you created to your dbt project by running the following commands in Terminal. Make sure you use the correct git URL for your repository, which you should have saved from step 5 in [Create a repository](./manual-install.md?step=2).

```shell
git init
git branch -M main
git add .
git commit -m "Create a dbt project"
git remote add origin https://github.com/USERNAME/dbt-tutorial.git
git push -u origin main
```

2. Return to your GitHub repository to verify your new files have been added.

### Build your first models

Now that you set up your sample project, you can get to the fun part — [building models](../docs/build/sql-models.md)! In the next steps, you will take a sample query and turn it into a model in your dbt project.

## Checkout a new git branch

Check out a new git branch to work on new code:

1. Create a new branch by using the `checkout` command and passing the `-b` flag:

```shell
$ git checkout -b add-customers-model
>  Switched to a new branch `add-customer-model`
```

## Build your first model

1. Open your project in your favorite code editor.
2. Create a new SQL file in the `models` directory, named `models/customers.sql`.
3. Paste the following query into the `models/customers.sql` file.

### BigQuery

```sql
with customers as (

    select
        id as customer_id,
        first_name,
        last_name

    from `dbt-tutorial`.jaffle_shop.customers

),

orders as (

    select
        id as order_id,
        user_id as customer_id,
        order_date,
        status

    from `dbt-tutorial`.jaffle_shop.orders

),

customer_orders as (

    select
        customer_id,

        min(order_date) as first_order_date,
        max(order_date) as most_recent_order_date,
        count(order_id) as number_of_orders

    from orders

    group by 1

),

final as (

    select
        customers.customer_id,
        customers.first_name,
        customers.last_name,
        customer_orders.first_order_date,
        customer_orders.most_recent_order_date,
        coalesce(customer_orders.number_of_orders, 0) as number_of_orders

    from customers

    left join customer_orders using (customer_id)

)

select * from final
```

### Databricks

```sql
with customers as (

    select
        id as customer_id,
        first_name,
        last_name

    from jaffle_shop_customers

),

orders as (

    select
        id as order_id,
        user_id as customer_id,
        order_date,
        status

    from jaffle_shop_orders

),

customer_orders as (

    select
        customer_id,

        min(order_date) as first_order_date,
        max(order_date) as most_recent_order_date,
        count(order_id) as number_of_orders

    from orders

    group by 1

),

final as (

    select
        customers.customer_id,
        customers.first_name,
        customers.last_name,
        customer_orders.first_order_date,
        customer_orders.most_recent_order_date,
        coalesce(customer_orders.number_of_orders, 0) as number_of_orders

    from customers

    left join customer_orders using (customer_id)

)

select * from final
```

### Redshift

```sql
with customers as (

    select
        id as customer_id,
        first_name,
        last_name

    from jaffle_shop.customers

),

orders as (

    select
        id as order_id,
        user_id as customer_id,
        order_date,
        status

    from jaffle_shop.orders

),

customer_orders as (

    select
        customer_id,

        min(order_date) as first_order_date,
        max(order_date) as most_recent_order_date,
        count(order_id) as number_of_orders

    from orders

    group by 1

),

final as (

    select
        customers.customer_id,
        customers.first_name,
        customers.last_name,
        customer_orders.first_order_date,
        customer_orders.most_recent_order_date,
        coalesce(customer_orders.number_of_orders, 0) as number_of_orders

    from customers

    left join customer_orders using (customer_id)

)

select * from final
```

### Snowflake

```sql
with customers as (

    select
        id as customer_id,
        first_name,
        last_name

    from raw.jaffle_shop.customers

),

orders as (

    select
        id as order_id,
        user_id as customer_id,
        order_date,
        status

    from raw.jaffle_shop.orders

),

customer_orders as (

    select
        customer_id,

        min(order_date) as first_order_date,
        max(order_date) as most_recent_order_date,
        count(order_id) as number_of_orders

    from orders

    group by 1

),

final as (

    select
        customers.customer_id,
        customers.first_name,
        customers.last_name,
        customer_orders.first_order_date,
        customer_orders.most_recent_order_date,
        coalesce(customer_orders.number_of_orders, 0) as number_of_orders

    from customers

    left join customer_orders using (customer_id)

)

select * from final
```

4. From the command line, enter `dbt run`.

[![A successful run with the dbt Core CLI](/img/first-model-dbt-cli.png?v=2 "A successful run with the dbt Core CLI")](#)A successful run with the dbt Core CLI

When you return to the BigQuery console, you can `select` from this model.

### FAQs

How can I see the SQL that dbt is running?

To check out the SQL that dbt is running, you can look in:

* dbt:
  * Within the run output, click on a model name, and then select "Details"

* dbt Core:

  * The `target/compiled/` directory for compiled `select` statements
  * The `target/run/` directory for compiled `create` statements
  * The `logs/dbt.log` file for verbose logging.

How did dbt choose which schema to build my models in?

By default, dbt builds models in your target schema. To change your target schema:

* If you're developing in **dbt**, these are set for each user when you first use a development environment.
* If you're developing with **dbt Core**, this is the `schema:` parameter in your `profiles.yml` file.

If you wish to split your models across multiple schemas, check out the docs on [using custom schemas](../docs/build/custom-schemas.md).

Note: on BigQuery, `dataset` is used interchangeably with `schema`.

Do I need to create my target schema before running dbt?

Nope! dbt will check if the schema exists when it runs. If the schema does not exist, dbt will create it for you.

If I rerun dbt, will there be any downtime as models are rebuilt?

Nope! The SQL that dbt generates behind the scenes ensures that any relations are replaced atomically (i.e. your business users won't experience any downtime).

The implementation of this varies on each warehouse, check out the [logs](../faqs/Runs/checking-logs.md) to see the SQL dbt is executing.

What happens if the SQL in my query is bad or I get a database error?

If there's a mistake in your SQL, dbt will return the error that your database returns.

```shell
$ dbt run --select customers
Running with dbt=1.9.0
Found 3 models, 9 tests, 0 snapshots, 0 analyses, 133 macros, 0 operations, 0 seed files, 0 sources

14:04:12 | Concurrency: 1 threads (target='dev')
14:04:12 |
14:04:12 | 1 of 1 START view model dbt_alice.customers.......................... [RUN]
14:04:13 | 1 of 1 ERROR creating view model dbt_alice.customers................. [ERROR in 0.81s]
14:04:13 |
14:04:13 | Finished running 1 view model in 1.68s.

Completed with 1 error and 0 warnings:

Database Error in model customers (models/customers.sql)
  Syntax error: Expected ")" but got identifier `your-info-12345` at [13:15]
  compiled SQL at target/run/jaffle_shop/customers.sql

Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
```

Any models downstream of this model will also be skipped. Use the error message and the [compiled SQL](../faqs/Runs/checking-logs.md) to debug any errors.

## Change the way your model is materialized

One of the most powerful features of dbt is that you can change the way a model is materialized in your warehouse, simply by changing a configuration value. You can change things between tables and views by changing a keyword rather than writing the data definition language (DDL) to do this behind the scenes.

By default, everything gets created as a view. You can override that at the directory level so everything in that directory will materialize to a different materialization.

1. Edit your `dbt_project.yml` file.

   * Update your project `name` to:

     dbt\_project.yml

     ```yaml
     name: 'jaffle_shop'
     ```

   * Configure `jaffle_shop` so everything in it will be materialized as a table; and configure `example` so everything in it will be materialized as a view. Update your `models` config in the project YAML file to:

     dbt\_project.yml

     ```yaml
     models:
       jaffle_shop:
         +materialized: table
         example:
           +materialized: view
     ```

   * Click **Save**.

2. Enter the `dbt run` command. Your `customers` model should now be built as a table!

   info

   To do this, dbt had to first run a `drop view` statement (or API call on BigQuery), then a `create table as` statement.

3. Edit `models/customers.sql` to override the `dbt_project.yml` for the `customers` model only by adding the following snippet to the top, and click **Save**:

   models/customers.sql

   ```sql
   {{
     config(
       materialized='view'
     )
   }}

   with customers as (

       select
           id as customer_id
           ...

   )
   ```

4. Enter the `dbt run` command. Your model, `customers`, should now build as a view.

   * BigQuery users need to run `dbt run --full-refresh` instead of `dbt run` to full apply materialization changes.

5. Enter the `dbt run --full-refresh` command for this to take effect in your warehouse.

### FAQs

What materializations are available in dbt?

dbt ships with five built-in materializations: `view`, `table`, `incremental`, `ephemeral`, and `materialized_view`. Check out the documentation on [materializations](../docs/build/materializations.md) for more information on each of these options.

You can also create your own [custom materializations](./create-new-materializations.md). This is an advanced feature of dbt.

Which materialization should I use for my model?

Start out with views, and then change models to tables when required for performance reasons (i.e. downstream queries have slowed).

Check out the [docs on materializations](../docs/build/materializations.md) for advice on when to use each materialization.

What model configurations exist?

You can also configure:

* [tags](../reference/resource-configs/tags.md) to support easy categorization and graph selection
* [custom schemas](../reference/resource-properties/schema.md) to split your models across multiple schemas
* [aliases](../reference/resource-configs/alias.md) if your view/table name should differ from the filename
* Snippets of SQL to run at the start or end of a model, known as [hooks](../docs/build/hooks-operations.md)
* Warehouse-specific configurations for performance (e.g. `sort` and `dist` keys on Redshift, `partitions` on BigQuery)

Check out the docs on [model configurations](../reference/model-configs.md) to learn more.

## Delete the example models

You can now delete the files that dbt created when you initialized the project:

1. Delete the `models/example/` directory.

2. Delete the `example:` key from your `dbt_project.yml` file, and any configurations that are listed under it.

   dbt\_project.yml

   ```yaml
   # before
   models:
     jaffle_shop:
       +materialized: table
       example:
         +materialized: view
   ```

   dbt\_project.yml

   ```yaml
   # after
   models:
     jaffle_shop:
       +materialized: table
   ```

3. Save your changes.

#### FAQs

How do I remove deleted models from my data warehouse?

If you delete a model from your dbt project, dbt does not automatically drop the relation from your schema. This means that you can end up with extra objects in schemas that dbt creates, which can be confusing to other users.

(This can also happen when you switch a model from being a view or table, to ephemeral)

When you remove models from your dbt project, you should manually drop the related relations from your schema.

I got an "unused model configurations" error message, what does this mean?

You might have forgotten to nest your configurations under your project name, or you might be trying to apply configurations to a directory that doesn't exist.

Check out this [article](https://discourse.getdbt.com/t/faq-i-got-an-unused-model-configurations-error-message-what-does-this-mean/112) to understand more.

## Build models on top of other models

As a best practice in SQL, you should separate logic that cleans up your data from logic that transforms your data. You have already started doing this in the existing query by using common table expressions (CTEs).

Now you can experiment by separating the logic out into separate models and using the [ref](../reference/dbt-jinja-functions/ref.md) function to build models on top of other models:

[![The DAG we want for our dbt project](/img/dbt-dag.png?v=2 "The DAG we want for our dbt project")](#)The DAG we want for our dbt project

1. Create a new SQL file, `models/stg_customers.sql`, with the SQL from the `customers` CTE in our original query.
2. Create a second new SQL file, `models/stg_orders.sql`, with the SQL from the `orders` CTE in our original query.

### BigQuery

models/stg\_customers.sql

```sql
select
    id as customer_id,
    first_name,
    last_name

from `dbt-tutorial`.jaffle_shop.customers
```

models/stg\_orders.sql

```sql
select
    id as order_id,
    user_id as customer_id,
    order_date,
    status

from `dbt-tutorial`.jaffle_shop.orders
```

### Databricks

models/stg\_customers.sql

```sql
select
    id as customer_id,
    first_name,
    last_name

from jaffle_shop_customers
```

models/stg\_orders.sql

```sql
select
    id as order_id,
    user_id as customer_id,
    order_date,
    status

from jaffle_shop_orders
```

### Redshift

models/stg\_customers.sql

```sql
select
    id as customer_id,
    first_name,
    last_name

from jaffle_shop.customers
```

models/stg\_orders.sql

```sql
select
    id as order_id,
    user_id as customer_id,
    order_date,
    status

from jaffle_shop.orders
```

### Snowflake

models/stg\_customers.sql

```sql
select
    id as customer_id,
    first_name,
    last_name

from raw.jaffle_shop.customers
```

models/stg\_orders.sql

```sql
select
    id as order_id,
    user_id as customer_id,
    order_date,
    status

from raw.jaffle_shop.orders
```

3. Edit the SQL in your `models/customers.sql` file as follows:

models/customers.sql

```sql
with customers as (

    select * from {{ ref('stg_customers') }}

),

orders as (

    select * from {{ ref('stg_orders') }}

),

customer_orders as (

    select
        customer_id,

        min(order_date) as first_order_date,
        max(order_date) as most_recent_order_date,
        count(order_id) as number_of_orders

    from orders

    group by 1

),

final as (

    select
        customers.customer_id,
        customers.first_name,
        customers.last_name,
        customer_orders.first_order_date,
        customer_orders.most_recent_order_date,
        coalesce(customer_orders.number_of_orders, 0) as number_of_orders

    from customers

    left join customer_orders using (customer_id)

)

select * from final
```

4. Execute `dbt run`.

This time, when you performed a `dbt run`, separate views/tables were created for `stg_customers`, `stg_orders` and `customers`. dbt inferred the order to run these models. Because `customers` depends on `stg_customers` and `stg_orders`, dbt builds `customers` last. You do not need to explicitly define these dependencies.

### FAQs

How do I run one model at a time?

To run one model, use the `--select` flag (or `-s` flag), followed by the name of the model:

```shell
$ dbt run --select customers
```

Check out the [model selection syntax documentation](../reference/node-selection/syntax.md) for more operators and examples.

Do ref-able resource names need to be unique?

Within one project: yes! To build dependencies between resources (such as models, seeds, and snapshots), you need to use the `ref` function, and pass in the resource name as an argument. dbt uses that resource name to uniquely resolve the `ref` to a specific resource. As a result, these resource names need to be unique, *even if they are in distinct folders*.

A resource in one project can have the same name as a resource in another project (installed as a dependency). dbt uses the project name to uniquely identify each resource. We call this "namespacing." If you `ref` a resource with a duplicated name, it will resolve to the resource within the same namespace (package or project), or raise an error because of an ambiguous reference. Use [two-argument `ref`](../reference/dbt-jinja-functions/ref.md#ref-project-specific-models) to disambiguate references by specifying the namespace.

Those resource will still need to land in distinct locations in the data warehouse. Read the docs on [custom aliases](../docs/build/custom-aliases.md) and [custom schemas](../docs/build/custom-schemas.md) for details on how to achieve this.

As I create more models, how should I keep my project organized? What should I name my models?

There's no one best way to structure a project! Every organization is unique.

If you're just getting started, check out how we (dbt Labs) [structure our dbt projects](../best-practices/how-we-structure/1-guide-overview.md).

### Next steps

Before moving on from building your first models, make a change and see how it affects your results:

* Write some bad SQL to cause an error — can you debug the error?

* Run only a single model at a time. For more information, see [Syntax overview](../reference/node-selection/syntax.md).

* Group your models with a `stg_` prefix into a `staging` subdirectory. For example, `models/staging/stg_customers.sql`.

  * Configure your `staging` models to be views.
  * Run only the `staging` models.

You can also explore:

* The `target` directory to see all of the compiled SQL. The `run` directory shows the create or replace table statements that are running, which are the select statements wrapped in the correct DDL.
* The `logs` file to see how dbt Core logs all of the action happening within your project. It shows the select statements that are running and the python logging happening when dbt runs.

## Add tests to your models

Adding [data tests](../docs/build/data-tests.md) to a project helps validate that your models are working correctly.

To add data tests to your project:

1. Create a new YAML file in the `models` directory, named `models/schema.yml`

2. Add the following contents to the file:

   models/schema.yml

   ```yaml
   version: 2

   models:
     - name: customers
       columns:
         - name: customer_id
           data_tests:
             - unique
             - not_null

     - name: stg_customers
       columns:
         - name: customer_id
           data_tests:
             - unique
             - not_null

     - name: stg_orders
       columns:
         - name: order_id
           data_tests:
             - unique
             - not_null
         - name: status
           data_tests:
             - accepted_values:
                 arguments: # available in v1.10.5 and higher. Older versions can set the <argument_name> as the top-level property.
                   values: ['placed', 'shipped', 'completed', 'return_pending', 'returned']
         - name: customer_id
           data_tests:
             - not_null
             - relationships:
                 arguments:
                   to: ref('stg_customers')
                   field: customer_id
   ```

3. Run `dbt test`, and confirm that all your tests passed.

When you run `dbt test`, dbt iterates through your YAML files, and constructs a query for each test. Each query will return the number of records that fail the test. If this number is 0, then the test is successful.

#### FAQs

What tests are available for me to use in dbt? Can I add my own custom tests?

Out of the box, dbt ships with the following data tests:

* `unique`
* `not_null`
* `accepted_values`
* `relationships` (for example, referential integrity)

You can also write your own [custom generic tests](../docs/build/data-tests.md#generic-data-tests).

Some additional generic tests have been open-sourced in the [dbt-utils package](https://github.com/dbt-labs/dbt-utils#generic-tests). Check out the docs on [packages](../docs/build/packages.md) to learn how to make these tests available in your project.

How do I test one model at a time?

Running tests on one model looks very similar to running a model: use the `--select` flag (or `-s` flag), followed by the name of the model:

```shell
dbt test --select customers
```

Check out the [model selection syntax documentation](../reference/node-selection/syntax.md) for full syntax, and [test selection examples](../reference/node-selection/test-selection-examples.md) in particular.

One of my tests failed, how can I debug it?

To debug a failing test, find the SQL that dbt ran by:

* dbt:

  * Within the test output, click on the failed test, and then select "Details".

* dbt Core:

  * Open the file path returned as part of the error message.
  * Navigate to the `target/compiled/schema_tests` directory for all compiled test queries.

Copy the SQL into a query editor (in dbt, you can paste it into a new `Statement`), and run the query to find the records that failed.

Does my test file need to be named \`schema.yml\`?

No! You can name this file whatever you want (including `whatever_you_want.yml`), so long as:

* The file is in your `models/` directory¹
* The file has `.yml` extension

Check out the [docs](../reference/configs-and-properties.md) for more information.

¹If you're declaring properties for seeds, snapshots, or macros, you can also place this file in the related directory — `seeds/`, `snapshots/` and `macros/` respectively.

Why do model and source YAML files always start with \`version: 2\`?

Once upon a time, the structure of these `.yml` files was very different (s/o to anyone who was using dbt back then!). Adding `version: 2` allowed us to make this structure more extensible.

From [dbt Core v1.5](<https://docs.getdbt.com/docs/dbt-versions/core-upgrade/Older versions/upgrading-to-v1.5.md#quick-hits>), the top-level `version:` key is optional in all resource YAML files. If present, only `version: 2` is supported.

Also starting in v1.5, both the [`config-version: 2`](../reference/project-configs/config-version.md) and the top-level `version:` key in the `dbt_project.yml` are optional.

Resource YAML files do not currently require this config. We only support `version: 2` if it's specified. Although we do not expect to update YAML files to `version: 3` soon, having this config will make it easier for us to introduce new structures in the future

What data tests should I add to my project?

We recommend that every model has a data test on a primary key, that is, a column that is `unique` and `not_null`.

We also recommend that you test any assumptions on your source data. For example, if you believe that your payments can only be one of three payment methods, you should test that assumption regularly — a new payment method may introduce logic errors in your SQL.

In advanced dbt projects, we recommend using [sources](../docs/build/sources.md) and running these source data-integrity tests against the sources rather than models.

When should I run my data tests?

You should run your data tests whenever you are writing new code (to ensure you haven't broken any existing models by changing SQL), and whenever you run your transformations in production (to ensure that your assumptions about your source data are still valid).

## Document your models

Adding [documentation](../docs/build/documentation.md) to your project allows you to describe your models in rich detail, and share that information with your team. Here, we're going to add some basic documentation to our project.

Update your `models/schema.yml` file to include some descriptions, such as those below.

models/schema.yml

```yaml
version: 2

models:
  - name: customers
    description: One record per customer
    columns:
      - name: customer_id
        description: Primary key
        data_tests:
          - unique
          - not_null
      - name: first_order_date
        description: NULL when a customer has not yet placed an order.

  - name: stg_customers
    description: This model cleans up customer data
    columns:
      - name: customer_id
        description: Primary key
        data_tests:
          - unique
          - not_null

  - name: stg_orders
    description: This model cleans up order data
    columns:
      - name: order_id
        description: Primary key
        data_tests:
          - unique
          - not_null
      - name: status
        data_tests:
          - accepted_values:
              arguments: # available in v1.10.5 and higher. Older versions can set the <argument_name> as the top-level property.
                values: ['placed', 'shipped', 'completed', 'return_pending', 'returned']
      - name: customer_id
        data_tests:
          - not_null
          - relationships:
              arguments:
                to: ref('stg_customers')
                field: customer_id
```

### View in Catalog

[Catalog](../docs/explore/explore-projects.md) provides powerful tools to interact with your dbt projects, including documentation:

1. Run one of the following commands:

   * `dbt docs generate` if you're on dbt Core
   * `dbt build` if you're on the dbt Fusion engine

2. Click **Catalog** in the navigation menu to launch Catalog.

3. Catalog reflects **Production** by default. If your account has additional environments (for example, **Staging**), you can select them from the environment dropdown.

[![Select an environment in Catalog.](/img/docs/collaborate/dbt-explorer/catalog-nav-dropdown.png?v=2 "Select an environment in Catalog.")](#)Select an environment in Catalog.

4. Select your project from the file tree.
5. Use the search bar or browse the resource list to find the `customers` model.
6. Click the model to view its details, including the descriptions you added.

[![View your model's documentation and lineage in Catalog.](/img/docs/collaborate/dbt-explorer/example-model-details.png?v=2 "View your model's documentation and lineage in Catalog.")](#)View your model's documentation and lineage in Catalog.

Catalog displays your model's description, column documentation, data tests, and lineage graph. You can also see which columns are missing documentation and track test coverage across your project.

### View in Studio IDE

You can view docs directly from the IDE if you're on `Latest` or another version of dbt Core. Keep in mind that this is a legacy view and doesn't offer the same level of interactivity as Catalog.

1. In the IDE, run `dbt docs generate`.
2. From the navigation bar, click the **View docs** icon located to the right of the **branch name**.

   [![The View docs icon in the Studio IDE.](/img/docs/collaborate/dbt-explorer/docs-icon.png?v=2 "The View docs icon in the Studio IDE.")](#)The View docs icon in the Studio IDE.
3. From **Projects**, select your project name and expand the folders.
4. Click **models** > **marts** > **customers**.

[![View your model's documentation in the legacy docs view.](/img/docs/collaborate/dbt-explorer/legacy-docs-view.png?v=2 "View your model's documentation in the legacy docs view.")](#)View your model's documentation in the legacy docs view.

3. Run `dbt docs serve` command to launch the documentation in a local website.

#### FAQs

How do I write long-form explanations in my descriptions?

If you need more than a sentence to explain a model, you can:

1. Split your description over multiple lines using `>`. Interior line breaks are removed and Markdown can be used. This method is recommended for simple, single-paragraph descriptions:

```yml
models:
  - name: customers
    description: >
      Lorem ipsum **dolor** sit amet, consectetur adipisicing elit, sed do eiusmod
      tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
      quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
      consequat.
```

2. Split your description over multiple lines using `|`. Interior line breaks are maintained and Markdown can be used. This method is recommended for more complex descriptions:

```yml
models:
  - name: customers
    description: |
      ### Lorem ipsum

      * dolor sit amet, consectetur adipisicing elit, sed do eiusmod
      * tempor incididunt ut labore et dolore magna aliqua.
```

3. Use a [docs block](../docs/build/documentation.md#using-docs-blocks) to write the description in a separate Markdown file.

How do I access documentation in dbt Catalog?

If you're using dbt to deploy your project and have a [Starter, Enterprise, or Enterprise+ plan](https://www.getdbt.com/pricing/), you can use Catalog to view your project's [resources](../docs/build/projects.md) (such as models, tests, and metrics) and their lineage to gain a better understanding of its latest production state.

Access Catalog in dbt by clicking the **Catalog** link in the navigation. You can have up to 5 read-only users access the documentation for your project.

dbt developer plan and dbt Core users can use [dbt Docs](../docs/explore/build-and-view-your-docs.md#dbt-docs), which generates basic documentation but it doesn't offer the same speed, metadata, or visibility as Catalog.

#### Next steps

Before moving on from testing, make a change and see how it affects your results:

* Write a test that fails, for example, omit one of the order statuses in the `accepted_values` list. What does a failing test look like? Can you debug the failure?
* Run the tests for one model only. If you grouped your `stg_` models into a directory, try running the tests for all the models in that directory.
* Use a [docs block](../docs/build/documentation.md#using-docs-blocks) to add a Markdown description to a model.

## Commit updated changes

You need to commit the changes you made to the project so that the repository has your latest code.

1. Add all your changes to git: `git add -A`
2. Commit your changes: `git commit -m "Add customers model, tests, docs"`
3. Push your changes to your repository: `git push -u origin add-customers-model`
4. Navigate to your repository, and open a pull request to merge the code into your master branch.

## Schedule a job

We recommend using dbt as the easiest and most reliable way to [deploy jobs](../docs/deploy/deployments.md) and automate your dbt project in production.

For more info on how to get started, refer to [create and schedule jobs](../docs/deploy/deploy-jobs.md#create-and-schedule-jobs).

[![Overview of a dbt job run, which includes the job run details, trigger type, commit SHA, environment name, detailed run steps, logs, and more.](/img/docs/dbt-platform/deployment/run-overview.png?v=2 "Overview of a dbt job run, which includes the job run details, trigger type, commit SHA, environment name, detailed run steps, logs, and more.")](#)Overview of a dbt job run, which includes the job run details, trigger type, commit SHA, environment name, detailed run steps, logs, and more.

For more information about using dbt Core to schedule a job, refer [dbt airflow](https://docs.getdbt.com/blog/dbt-airflow-spiritual-alignment) blog post.

## Was this page helpful?

YesNo

[Privacy policy](https://www.getdbt.com/cloud/privacy-policy)[Create a GitHub issue](https://github.com/dbt-labs/docs.getdbt.com/issues)

This site is protected by reCAPTCHA and the Google [Privacy Policy](https://policies.google.com/privacy) and [Terms of Service](https://policies.google.com/terms) apply.
