dbt Semantic Layer

Coming soon

The dbt Semantic Layer is undergoing some sophisticated changes, enabling more complex metric definitions and efficient querying. As part of these changes, the dbt_metrics package will be deprecated and replaced with MetricFlow. For more info, check out the The dbt Semantic Layer: what's next? and dbt_metrics deprecation blog.

The dbt Semantic Layer allows data teams to centrally define essential business metrics like revenue, customer, and churn in the modeling layer (your dbt project) for consistent self-service within downstream data tools like BI and metadata management solutions. The dbt Semantic Layer provides the flexibility to define metrics on top of your existing models and then query those metrics and models in your analysis tools of choice.

The result? You have less duplicative coding for data teams and more consistency for data consumers.

The dbt Semantic Layer has four main parts:

• Define your metrics in version-controlled dbt project code
• Import your metric definitions via the Discovery API
• Query your metric data via the dbt Proxy Server
• Explore and analyze dbt metrics in downstream tools

dbt Semantic Layer architecture

What makes the dbt Semantic Layer different?

The dbt Semantic Layer reduces code duplication and inconsistency regarding your business metrics. By moving metric definitions out of the BI layer and into the modeling layer, data teams can feel confident that different business units are working from the same metric definitions, regardless of their tool of choice. If a metric definition changes in dbt, it’s refreshed everywhere it’s invoked and creates consistency across all applications. You can also use the dbt Semantic Layer to query models and use macros.

Prerequisites

To use the dbt Semantic Layer, you’ll need to meet the following:

Considerations

Some important considerations to know about using the dbt Semantic Layer during the Public Preview:

• Support for Snowflake data platform only (additional data platforms coming soon)
• Support for the deployment environment only (development experience coming soon)

Public Preview

The dbt Semantic Layer is currently available for Public Preview, which means:

— Who? The dbt Semantic Layer is open to all dbt Cloud tiers (Developer, Team, and Enterprise) during Public Preview. Review Product architecture for more info on plan availability.

• Team and Enterprise accounts will be able to set up the Semantic Layer and Discovery API in the integrated partner tool to import metric definition.
• Developer accounts will be able to query the Proxy Server using SQL, but will not be able to browse dbt metrics in external tools, which requires access to the Discovery API.

— What? Public Previews provide early access to new features. The Semantic Layer is stable and you can use it for production deployments, but there may still be some planned additions and modifications to product behaviors before moving to General Availability. We may also introduce new functionality that is not backwards compatible. dbt Labs provides support, and relevant service level objectives (SLOs) apply. We will introduce pricing for the dbt Semantic Layer alongside the General Available (GA) release (future GA date to be announced).

— When? Public Preview will end once the dbt Semantic Layer is available for GA. After GA, the dbt Semantic Layer will only be available to dbt Cloud Team and Enterprise plans.

— Where? Public Preview is enabled at the account level so you don’t need to worry about enabling it per user.

Product architecture

The dbt Semantic Layer product architecture includes four primary components:

Components	Information	Developer plans	Team plans	Enterprise plans	License
dbt project	Define models and metrics in dbt Core.	✅	✅	✅	Open source, Core
dbt Server	A persisted HTTP server that wraps dbt core to handle RESTful API requests for dbt operations.	✅	✅	✅	BSL
SQL Proxy	Reverse-proxy that accepts dbt-SQL (SQL + Jinja like query models and metrics, use macros), compiles the query into pure SQL, and executes the query against the data platform.	✅ * Available during Public Preview only	✅	✅	Proprietary, Cloud (Team & Enterprise)
Discovery API	Accesses metric definitions primarily via integrations and is the source of truth for objects defined in dbt projects (like models, macros, sources, metrics). The Discovery API is updated at the end of every dbt Cloud run.	❌	✅	✅	Proprietary, Cloud (Team & Enterprise

dbt Semantic components

dbt Semantic Layer integrations will:

• Leverage the Discovery API to fetch a list of objects and their attributes, like metrics
• Generate a dbt-SQL statement
• Then query the SQL proxy to evaluate the results of this statement

Manage metrics

📌

New to dbt or metrics? Check out our quickstart guide to build your first dbt project! If you'd like to define your first metrics, try our Jaffle Shop example project.

If you're not sure whether to define a metric in dbt or not, ask yourself the following:

Is this something our teams consistently need to report on?

An important business metric should be:

• Well-defined (the definition is agreed upon throughout the entire organization)
• Time-bound (able to be compared across time)

A great example of this is revenue — it can be aggregated on multiple levels (weekly, monthly, etc) and is key for the broader business to understand.

• ✅ Monthly recurring revenue or Weekly active users or Average order value
• ❌ 1-off experimental metric

Design and define metrics

Design metrics To read about best practices on structuring and organizing your metrics, review our How to design and structure dbt metrics: Recommendations for getting started blog post first.

Define metrics You can define your metrics in .yml files nested under a metrics key and to design or define your own metrics in your dbt project, review the following documents:

• How to design and structure dbt metrics: Recommendations for getting started blog to understand best practices for designing and structuring metrics in your dbt project
• dbt metrics for in-depth detail on attributes, filters, how to define and query your metrics and dbt-metrics package
• dbt Semantic Layer quickstart to get started
• Understanding the components of the dbt Semantic Layer blog post to see further examples

Review our helpful metrics video below, which explains what metrics are, why they're important and how you can get started:

Related questions

How are you storing my data?

The dbt Semantic Layer does not store, or cache, or log your data. On each query to the Semantic Layer, the resulting data passes through dbt Cloud servers where it is never stored, cached, or logged. The data from your data platform gets routed through dbt Cloud servers, to your connecting data tool.

Is the dbt Semantic Layer open source?

Some components of the dbt Semantic Layer are open source like dbt-core, the dbt_metrics package, and the BSL licensed dbt-server. The dbt Proxy Server (what is actually compiling the dbt code) and the Discovery API are not open source.

During Public Preview, the dbt Semantic Layer is open to all dbt Cloud tiers (Developer, Team, and Enterprise).

• dbt Core users can define metrics in their dbt Core projects and calculate them using macros from the metrics package. To use the dbt Semantic Layer integrations, users will need to have a dbt Cloud account.


• Developer accounts will be able to query the Proxy Server using SQL, but will not be able to browse pre-populated dbt metrics in external tools, which requires access to the Discovery API.


• Team and Enterprise accounts will be able to set up the Semantic Layer and Discovery API in the integrated partner tool to import metric definition.

Is there a dbt Semantic Layer discussion hub?

Yes absolutely! Join the dbt Slack community and #dbt-cloud-semantic-layer slack channel for all things related to the dbt Semantic Layer.