graph

The graph context variable contains information about the nodes in your dbt project. Models, sources, tests, and snapshots are all examples of nodes in dbt projects.

Heads up

dbt actively builds the graph variable during the parsing phase of running dbt projects, so some properties of the graph context variable will be missing or incorrect during parsing. Please read the information below carefully to understand how to effectively use this variable.

The graph context variable

Changelog

The graph context variable is a dictionary which maps node ids onto dictionary representations of those nodes. A simplified example might look like:

{
"nodes": {
"model.project_name.model_name": {
"config": {"materialized": "table", "sort": "id"},
"tags": ["abc", "123"],
"path": "models/path/to/model_name.sql",
...
},
},
"sources": {
"source.project_name.snowplow.event": {
"database": "analytics",
"schema": "analytics",
"unique_id": "source.project_name.snowplow.event",
"tags": ["abc", "123"],
"path": "models/path/to/schema.yml",
...
},
}

The exact contract for these model and source nodes is not currently documented, but that will change in the future.

Accessing models

The model entries in the graph dictionary will be incomplete or incorrect during parsing. If accessing the models in your project via the graph variable, be sure to use the execute flag to ensure that this code only executes at run-time and not at parse-time. Do not use the graph variable to build you DAG, as the resulting dbt behavior will be undefined and likely incorrect. Example usage:

graph-usage.sql
/*
Print information about all of the models in the Snowplow package
*/
{% if execute %}
{% for node in graph.nodes.values()
| selectattr("resource_type", "equalto", "model")
| selectattr("package_name", "equalto", "snowplow") %}
{% do log(node.unique_id ~ ", materialized: " ~ node.config.materialized, info=true) %}
{% endfor %}
{% endif %}
/*
Example output
---------------------------------------------------------------
model.snowplow.snowplow_id_map, materialized: incremental
model.snowplow.snowplow_page_views, materialized: incremental
model.snowplow.snowplow_web_events, materialized: incremental
model.snowplow.snowplow_web_page_context, materialized: table
model.snowplow.snowplow_web_events_scroll_depth, materialized: incremental
model.snowplow.snowplow_web_events_time, materialized: incremental
model.snowplow.snowplow_web_events_internal_fixed, materialized: ephemeral
model.snowplow.snowplow_base_web_page_context, materialized: ephemeral
model.snowplow.snowplow_base_events, materialized: ephemeral
model.snowplow.snowplow_sessions_tmp, materialized: incremental
model.snowplow.snowplow_sessions, materialized: table
*/

Accessing sources

To access the sources in your dbt project programatically, use the sources attribute of the graph object.

Example usage:

models/events_unioned.sql
/*
Union all of the Snowplow sources defined in the project
which begin with the string "event_"
*/
{% set sources = [] -%}
{% for node in graph.sources.values() -%}
{%- if node.name.startswith('event_') and node.source_name == 'snowplow' -%}
{%- do sources.append(source(node.source_name, node.name)) -%}
{%- endif -%}
{%- endfor %}
select * from (
{%- for source in sources %}
{{ source }} {% if not loop.last %} union all {% endif %}
{% endfor %}
)
/*
Example compiled SQL
---------------------------------------------------------------
select * from (
select * from raw.snowplow.event_add_to_cart union all
select * from raw.snowplow.event_remove_from_cart union all
select * from raw.snowplow.event_checkout
)
*/