Skip to main content

dbt Semantic Layer

📌

The dbt Semantic Layer is currently available in Public Preview for multi-tenant dbt Cloud accounts hosted in North America. If you log in via https://cloud.getdbt.com/, you can access the Semantic Layer. If you log in with another URL, the dbt Semantic Layer will be available in the future.

For more info, review the Prerequisites, Public Preview, and Product architecture sections.

The dbt Semantic Layer allows data teams to centrally define essential business metrics like revenuecustomer, and churn in the modeling layer (your dbt project) for consistent self-service within downstream data tools like BI and metadata management solutions.

The result? You have less duplicative coding for data teams and more consistency for data consumers.

The dbt Semantic Layer has four main parts:

  • Define your metrics in version-controlled dbt project code
  • Import your metric definitions via the Metadata API
  • Query your metric data via the dbt Proxy Server
  • Explore and analyze dbt metrics in downstream tools
dbt Semantic Layer architecture

dbt Semantic Layer architecture

What makes the dbt Semantic Layer different?

The dbt Semantic Layer reduces code duplication and inconsistency regarding your business metrics. By moving metric definitions out of the BI layer and into the modeling layer, data teams can feel confident that different business units are working from the same metric definitions, regardless of their tool of choice. If a metric definition changes in dbt, it’s refreshed everywhere it’s invoked and creates consistency across all applications.

Prerequisites

To use the dbt Semantic Layer, you’ll need to meet the following:

Considerations

Some important considerations to know about using the dbt Semantic Layer during the Public Preview:

  • Support for Snowflake data platform only (additional data platforms coming soon)
  • Support for the deployment environment only (development experience coming soon)

Public Preview

The dbt Semantic Layer is currently available for Public Preview, which means:

Who? The dbt Semantic Layer is open to all dbt Cloud tiers (Developer, Team, and Enterprise) during Public Preview. Review Product architecture for more info on plan availability.

  • Team and Enterprise accounts will be able to set up the Semantic Layer and Metadata API in the integrated partner tool to import metric definition.
  • Developer accounts will be able to query the Proxy Server using SQL, but will not be able to browse dbt metrics in external tools, which requires access to the Metadata API.

What? Public Previews provide early access to new features. The Semantic Layer is stable and you can use it for production deployments, but there may still be some planned additions and modifications to product behaviors before moving to General Availability. We may also introduce new functionality that is not backwards compatible. dbt Labs provides support, and relevant service level objectives (SLOs) apply. We will introduce pricing for the dbt Semantic Layer alongside the General Available (GA) release (future GA date to be announced).

When? Public Preview will end once the dbt Semantic Layer is available for GA. After GA, the dbt Semantic Layer will only be available to dbt Cloud Team and Enterprise plans.

Where? Public Preview is enabled at the account level so you don’t need to worry about enabling it per user.

Product architecture

The dbt Semantic Layer product architecture includes four primary components:

ComponentsInformationDeveloper plansTeam plansEnterprise plansLicense
dbt metricsAllows you to define metrics in dbt Core.Open source, Core
dbt ServerA persisted HTTP server that wraps dbt core to handle RESTful API requests for dbt operations.BSL
SQL ProxyReverse-proxy that accepts dbt-SQL (SQL + Jinja like query models and metrics, use macros), compiles the query into pure SQL, and executes the query against the data platform.
* Available during Public Preview only
Proprietary, Cloud (Team & Enterprise)
Metadata APIAccesses metric definitions primarily via integrations and is the source of truth for objects defined in dbt projects (like models, macros, sources, metrics). The Metadata API is updated at the end of every dbt Cloud run.Proprietary, Cloud (Team & Enterprise
dbt Semantic components

dbt Semantic components

dbt Semantic Layer integrations will:

  • Leverage the Metadata API to fetch a list of objects and their attributes, like metrics
  • Generate a dbt-SQL statement
  • Then query the SQL proxy to evaluate the results of this statement

Manage metrics

📌

New to dbt or metrics? Check out our Getting Started guide to build your first dbt project! If you'd like to define your first metrics, try our Jaffle Shop example project.

If you're not sure whether to define a metric in dbt or not, ask yourself the following:

Is this something our teams consistently need to report on?

An important business metric should be:

  • Well-defined (the definition is agreed upon throughout the entire organization)
  • Time-bound (able to be compared across time)

A great example of this is revenue it can be aggregated on multiple levels (weekly, monthly, etc) and is key for the broader business to understand.

  • Monthly recurring revenue or Weekly active users or Average order value
  • 1-off experimental metric

Design and define metrics

Design metrics To read about best practices on structuring and organizing your metrics, review our How to design and structure dbt metrics: Recommendations for getting started blog post first.

Define metrics You can define your metrics in .yml files nested under a metrics key and to design or define your own metrics in your dbt project, review the following documents:

Review our helpful metrics video below, which explains what metrics are, why they're important and how you can get started:

How are you storing my data?
The dbt Semantic Layer does not store, or cache, or log your data. On each query to the Semantic Layer, the resulting data passes through dbt Cloud servers where it is never stored, cached, or logged. The data from your data platform gets routed through dbt Cloud servers, to your connecting data tool.
Is the dbt Semantic Layer open source?
Some components of the dbt Semantic Layer are open source like dbt-core, the dbt_metrics package, and the BSL licensed dbt-server. The dbt Proxy Server (what is actually compiling the dbt code) and the Metadata API are not open source.

During Public Preview, the dbt Semantic Layer is open to all dbt Cloud tiers (Developer, Team, and Enterprise).

  • dbt Core users can define metrics in their dbt Core projects and calculate them using macros from the metrics package. To use the dbt Semantic Layer integrations, users will need to have a dbt Cloud account.

  • Developer accounts will be able to query the Proxy Server using SQL, but will not be able to browse pre-populated dbt metrics in external tools, which requires access to the Metadata API.

  • Team and Enterprise accounts will be able to set up the Semantic Layer and Metadata API in the integrated partner tool to import metric definition.
Is there a dbt Semantic Layer discussion hub?
Yes absolutely! Join the dbt Slack community and #dbt-cloud-semantic-layer slack channel for all things related to the dbt Semantic Layer.

0