Skip to main content

Explore your dbt projects

With dbt Explorer, you can view your project's resources (such as models, tests, and metrics) and their lineage to gain a better understanding of its latest production state. Navigate and manage your projects within dbt Cloud to help you and other data developers, analysts, and consumers discover and leverage your dbt resources.

Public preview

Try dbt Explorer! It's available in Public Preview as of October 17, 2023 for dbt Cloud customers. More updates coming soon.

Prerequisites

  • You have a multi-tenant or AWS single-tenant dbt Cloud account on the Team or Enterprise plan.
  • You have set up a production deployment environment for each project you want to explore.
    • There has been at least one successful job run in the production deployment environment.
  • You are on the dbt Explorer page. To do this, select Explore from the top navigation bar in dbt Cloud.

Generate metadata

dbt Explorer uses the metadata provided by the Discovery API to display the details about the state of your project. The metadata that's available depends on the deployment environment you've designated as production in your dbt Cloud project. dbt Explorer automatically retrieves the metadata updates after each job run in the production deployment environment so it always has the latest results for your project.

To view a resource and its metadata, you must define the resource in your project and run a job in the production environment. The resulting metadata depends on the commands executed by the jobs.

For a richer experience with dbt Explorer, you must:

  • Run dbt run or dbt build on a given model within a job in the environment to update model details or results.
  • Run dbt docs generate within a job in the environment to view catalog statistics and columns for models, sources, and snapshots.
  • Run dbt test or dbt build within a job in the environment to view test results.
  • Run dbt source freshness within a job in the environment to view source freshness data.
  • Run dbt snapshot or dbt build within a job in the environment to view snapshot details.

Richer and more timely metadata will become available as dbt Core, the Discovery API, and the underlying dbt Cloud platform evolves.

Explore your project's lineage graph

dbt Explorer provides a visualization of your project’s DAG that you can interact with. To access the project's full lineage graph, select Overview in the left sidebar and click the Explore Lineage button on the main (center) section of the page.

If you don't see the project lineage graph immediately, click Render Lineage. It can take some time for the graph to render depending on the size of your project and your computer’s available memory. The graph of very large projects might not render so you can select a subset of nodes by using selectors, instead.

The nodes in the lineage graph represent the project’s resources and the edges represent the relationships between the nodes. Nodes are color-coded and include iconography according to their resource type.

By default, dbt Explorer shows the project's applied state lineage. That is, it shows models that have been successfully built and are available to query, not just the models defined in the project.

To explore the lineage graphs of tests and macros, view their resource details pages. By default, dbt Explorer excludes these resources from the full lineage graph unless a search query returns them as results.

To interact with the full lineage graph, you can:

  • Hover over any item in the graph to display the resource’s name and type.

  • Zoom in and out on the graph by mouse-scrolling.

  • Grab and move the graph and the nodes.

  • Right click on a node (context menu) to:

    • Refocus on the node, including its parent and child nodes
    • Refocus on the node and its children only
    • Refocus on the node and it parents only
    • View the node's resource details page
  • Select a resource to highlight its relationship with other resources in your project. A panel opens on the graph’s right-hand side that displays a high-level summary of the resource’s details. The side panel includes a General tab for information like description, materialized type, and other details.

    • Click the Share icon in the side panel to copy the graph’s link to your clipboard.
    • Click the View Resource icon in the side panel to view the resource details.
  • Search and select specific resources or a subset of the DAG using selectors and graph operators. This can help you narrow the focus on the resources that interest you. For example:

    • +[RESOURCE_NAME] Displays all parent nodes of the resource
    • resource_type:model [RESOURCE_NAME] Displays all models matching the name search
  • View resource details by selecting a node (double-clicking) in the graph.

  • Click the List view icon in the graph's upper right corner to return to the main Explore page.

Search for resources

With the search bar (on the upper left corner of the page or in a lineage graph), you can search with keywords or selectors. The resources that match your search criteria will display in a list (keyword search) or the lineage graph (selector search).

Search with keywords

When searching with keywords, dbt Explorer searches through your resource metadata, such as resource name, and returns any matches. When searching for an exact column name, the results show all relational nodes containing that column in their schemas. If there's a match, a notice in the search result indicates the resource contains the specified column.

You can then refine the search results using the filters in the panel on the right-hand side of the page. Use this panel to select specific resource types or tags to apply. You can filter on model properties (access or materialization type) under the Models option.

Search with selectors

You can search with selectors within the full lineage graph using the search bar. Below are the selectors currently available in dbt Explorer:

  • fqn: Find resources by file or fully qualified name. This selector is the search bar's default. If you want to use the default, it's unnecessary to add fqn: before the search term.
  • source: Find resources by a specified source.
  • resource_type: Find resources by their type.
  • package: Find resources by the dbt package that defines them.
  • tag: Find resources by a specified tag.

When searching with selector methods, you can also use graph operators. For example, +orders returns all the upstream nodes of orders and orders+1 returns just the nodes immediately downstream (the children) of orders.

You can use multiple selector methods in your search query with set operators. A space implies a union set operator and a comma for an intersection. For example:

  • resource_type:metric,tag:nightly Returns metrics with the tag nightly
  • +snowplow_sessions +fct_orders Returns resources that are parent nodes of either snowplow_sessions or fct_orders

Browse with the sidebar

By default, the catalog sidebar lists all your project’s resources. Select any resource type in the list and all those resources in the project will display as a table in the main section of the page. For a description on the different resource types (like models, metrics, and so on), refer to About dbt projects.

To browse using a different view, you can choose one of these options from the View by dropdown:

  • Resources (default) All resources in the project organized by type.
  • Packages All resources in the project organized by the dbt package in which they are defined.
  • File Tree All resources in the project organized by the file in which they are defined. This mirrors the file tree in your dbt project repository.
  • Database All resources in the project organized by the database and schema in which they are built. This mirrors your data platform's structure that represents the applied state of your project.

View model versions

If models in the project are versioned, you can see which version of the model is being applied prerelease, latest, and old in the title of the model’s details page and in the model list from the sidebar.

View resource details

You can view the definition and latest run results of any resource in your project. To find a resource and view its details, you can interact with the lineage graph, use search, or browse the catalog.

The details (metadata) available to you depends on the resource’s type, its definition, and the commands that run within jobs in the production environment.

Example of model details

An example of the details you might get for a model:

  • Status bar (below the page title) Information on the last time the model ran, whether the run was successful, how the data is materialized, number of rows, and the size of the model.
  • General tab includes:
    • Lineage graph The model’s lineage graph that you can interact with. The graph includes one parent node and one child node from the model. Click the Expand icon in the graph's upper right corner to view the model in full lineage graph mode.
    • Description section A description of the model.
    • Recent section Information on the last time the model ran, how long it ran for, whether the run was successful, the job ID, and the run ID.
    • Tests section Tests for the model, including a status indicator for the latest test status. A ✅ denotes a passing test.
    • Details section Key properties like the model’s relation name (for example, how it’s represented and how you can query it in the data platform: database.schema.identifier); model governance attributes like access, group, and if contracted; and more.
    • Relationships section The nodes the model Depends On, is Referenced by, and (if applicable) is Used by for projects that have declared the models' project as a dependency.
  • Code tab The source code and compiled code for the model.
  • Columns tab The available columns in the model. This tab also shows tests results (if any) that you can select to view the test's details page. A ✅ denotes a passing test. To filter the columns in the resource, you can use the search bar that's located at the top of the columns view.

Example of exposure details

An example of the details you might get for an exposure:

  • Status bar (below the page title) Information on the last time the exposure was updated.
  • General tab includes:
    • Status section The status on data freshness and data quality.
    • Lineage graph The exposure’s lineage graph. Click the Expand icon in the graph's upper right corner to view the exposure in full lineage graph mode.
    • Description section A description of the exposure.
    • Details section Details like exposure type, maturity, owner information, and more.
    • Relationships section The nodes the exposure Depends On.

Example of test details

An example of the details you might get for a test:

  • Status bar (below the page title) Information on the last time the test ran, whether the test passed, test name, test target, and column name.
  • General tab includes:
    • Lineage graph The test’s lineage graph that you can interact with. The graph includes one parent node and one child node from the test resource. Click the Expand icon in the graph's upper right corner to view the test in full lineage graph mode.
    • Description section A description of the test.
    • Recent section Information on the last time the test ran, how long it ran for, whether the test passed, the job ID, and the run ID.
    • Details section Details like schema, severity, package, and more.
    • Relationships section The nodes the test Depends On.
  • Code tab The source code and compiled code for the test.

Example of source details

An example of the details you might get for each source table within a source collection:

  • Status bar (below the page title) Information on the last time the source was updated and the number of tables the source uses.
  • General tab includes:
    • Lineage graph The source’s lineage graph that you can interact with. The graph includes one parent node and one child node from the source. Click the Expand icon in the graph's upper right corner to view the source in full lineage graph mode.
    • Description section A description of the source.
    • Source freshness section Information on whether refreshing the data was successful, the last time the source was loaded, the timestamp of when a run generated data, and the run ID.
    • Details section Details like database, schema, and more.
    • Relationships section A table that lists all the sources used with their freshness status, the timestamp of when freshness was last checked, and the timestamp of when the source was last loaded.
  • Columns tab The available columns in the source. This tab also shows tests results (if any) that you can select to view the test's details page. A ✅ denotes a passing test.
0