Skip to main content

YAML Selectors

Write resource selectors in YAML, save them with a human-friendly name, and reference them using the --selector flag. By recording selectors in a top-level selectors.yml file:

  • Legibility: complex selection criteria are composed of dictionaries and arrays
  • Version control: selector definitions are stored in the same git repository as the dbt project
  • Reusability: selectors can be referenced in multiple job definitions, and their definitions are extensible (via YAML anchors)

Selectors live in a top-level file named selectors.yml. Each must have a name and a definition, and can optionally define a description and default flag.

selectors.yml
selectors:
- name: nodes_to_joy
definition: ...
- name: nodes_to_a_grecian_urn
description: Attic shape with a fair attitude
default: true
definition: ...

Definitions

Each definition is comprised of one or more arguments, which can be one of the following:

  • CLI-style: strings, representing CLI-style arguments
  • Key-value: pairs in the form method: value
  • Full YAML: fully specified dictionaries with items for method, value, operator-equivalent keywords, and support for exclude

Use the union and intersection operator-equivalent keywords to organize multiple arguments.

CLI-style

definition:
'tag:nightly'

This simple syntax supports use of the +, @, and * graph operators, but it does not support set operators or exclude.

Key-value

definition:
tag: nightly

This simple syntax does not support any graph or set operators or exclude.

Full YAML

This is the most thorough syntax, which can include the operator-equivalent keywords for graph and set operators.

Review methods for the available list.

The * operator to select all nodes can be written as:

definition:
method: fqn
value: "*"

Exclude

The exclude keyword is only supported by fully-qualified dictionaries. It may be passed as an argument to each dictionary, or as an item in a union. The following are equivalent:

- method: tag
value: nightly
exclude:
- "@tag:daily"
- union:
- method: tag
value: nightly
- exclude:
- method: tag
value: daily

Note: The exclude argument in YAML selectors is subtly different from the --exclude CLI argument. Here, exclude always returns a set difference, and it is always applied last within its scope.

Indirect selection

See test selection examples for more details about indirect selection.

Example

Here are two ways to represent:

$ dbt run --select @source:snowplow,tag:nightly models/export --exclude package:snowplow,config.materialized:incremental export_performance_timing
selectors.yml

selectors:
- name: nightly_diet_snowplow
description: "Non-incremental Snowplow models that power nightly exports"
definition:

# Optional `union` and `intersection` keywords map to the ` ` and `,` set operators:
union:
- intersection:
- '@source:snowplow'
- 'tag:nightly'
- 'models/export'
- exclude:
- intersection:
- 'package:snowplow'
- 'config.materialized:incremental'
- export_performance_timing

Then in our job definition:

dbt run --selector nightly_diet_snowplow

Default

Selectors may define a boolean default property. If a selector has default: true, dbt will use this selector's criteria when tasks do not define their own selection criteria.

Let's say we define a default selector that only selects resources defined in our root project:

selectors:
- name: root_project_only
description: >
Only resources from the root project.
Excludes resources defined in installed packages.
default: true
definition:
method: package
value: <my_root_project_name>

If I run an "unqualified" command, dbt will use the selection criteria defined in root_project_only—that is, dbt will only build / freshness check / generate compiled SQL for resources defined in my root project.

dbt build
dbt source freshness
dbt docs generate

If I run a command that defines its own selection criteria (via --select, --exclude, or --selector), dbt will ignore the default selector and use the flag criteria instead. It will not try to combine the two.

dbt run --select  "model_a"
dbt run --exclude model_a

Only one selector may set default: true for a given invocation; otherwise, dbt will return an error. You may use a Jinja expression to adjust the value of default depending on the environment, however:

selectors:
- name: default_for_dev
default: "{{ target.name == 'dev' | as_bool }}"
definition: ...
- name: default_for_prod
default: "{{ target.name == 'prod' | as_bool }}"
definition: ...
0