Skip to main content

YAML Selectors

Write resource selectors in YAML, save them with a human-friendly name, and reference them using the --selector flag. By recording selectors in a top-level selectors.yml file:

  • Legibility: complex selection criteria are composed of dictionaries and arrays
  • Version control: selector definitions are stored in the same git repository as the dbt project
  • Reusability: selectors can be referenced in multiple job definitions, and their definitions are extensible (via YAML anchors)

Selectors live in a top-level file named selectors.yml. Each must have a name and a definition, and can optionally define a description and default flag.

selectors.yml
selectors:
  - name: nodes_to_joy
    definition: ...
  - name: nodes_to_a_grecian_urn
    description: Attic shape with a fair attitude
    default: true
    definition: ...

Definitions​

Each definition is comprised of one or more arguments, which can be one of the following:

  • CLI-style: strings, representing CLI-style arguments
  • Key-value: pairs in the form method: value
  • Full YAML: fully specified dictionaries with items for method, value, operator-equivalent keywords, and support for exclude

Use the union and intersection operator-equivalent keywords to organize multiple arguments.

CLI-style​

definition:
  'tag:nightly'

This simple syntax supports use of the +, @, and * graph operators, but it does not support set operators or exclude.

Key-value​

definition:
  tag: nightly

This simple syntax does not support any graph or set operators or exclude.

Full YAML​

This is the most thorough syntax, which can include the operator-equivalent keywords for graph and set operators.

Review methods for the available list.

The * operator to select all nodes can be written as:

definition:
  method: fqn
  value: "*"

Exclude​

The exclude keyword is only supported by fully-qualified dictionaries. It may be passed as an argument to each dictionary, or as an item in a union. The following are equivalent:

- method: tag
  value: nightly
  exclude:
    - "@tag:daily"
- union:
    - method: tag
      value: nightly
    - exclude:
       - method: tag
         value: daily

Note: The exclude argument in YAML selectors is subtly different from the --exclude CLI argument. Here, exclude always returns a set difference, and it is always applied last within its scope.

Indirect selection​

See test selection examples for more details about indirect selection.

Example​

Here are two ways to represent:

$ dbt run --select @source:snowplow,tag:nightly models/export --exclude package:snowplow,config.materialized:incremental export_performance_timing
selectors.yml

selectors:
  - name: nightly_diet_snowplow
    description: "Non-incremental Snowplow models that power nightly exports"
    definition:

      # Optional `union` and `intersection` keywords map to the ` ` and `,` set operators:
      union:
        - intersection:
            - '@source:snowplow'
            - 'tag:nightly'
        - 'models/export'
        - exclude:
            - intersection:
                - 'package:snowplow'
                - 'config.materialized:incremental'
            - export_performance_timing

Then in our job definition:

$ dbt run --selector nightly_diet_snowplow

Default​

Selectors may define a boolean default property. If a selector has default: true, dbt will use this selector's criteria when tasks do not define their own selection criteria.

Let's say we define a default selector that only selects resources defined in our root project:

selectors:
  - name: root_project_only
    description: >
        Only resources from the root project.
        Excludes resources defined in installed packages.
    default: true
    definition:
      method: package
      value: <my_root_project_name>

If I run an "unqualified" command, dbt will use the selection criteria defined in root_project_only—that is, dbt will only build / freshness check / generate compiled SQL for resources defined in my root project.

$ dbt build
$ dbt source freshness
$ dbt docs generate

If I run a command that defines its own selection criteria (via --select, --exclude, or --selector), dbt will ignore the default selector and use the flag criteria instead. It will not try to combine the two.

$ dbt run --select  model_a
$ dbt run --exclude model_a

Only one selector may set default: true for a given invocation; otherwise, dbt will return an error. You may use a Jinja expression to adjust the value of default depending on the environment, however:

selectors:
  - name: default_for_dev
    default: "{{ target.name == 'dev' | as_bool }}"
    definition: ...
  - name: default_for_prod
    default: "{{ target.name == 'prod' | as_bool }}"
    definition: ...
0
Edit this page