Overview
With every invocation, dbt generates and saves one or more artifacts. Several of these are JSON files (manifest.json
, catalog.json
, run_results.json
, and sources.json
) that are used to power:
They could also be used to:
- calculate project-level test coverage
- perform longitudinal analysis of run timing
- identify historical changes in table structure
- do much, much more
dbt has produced artifacts since the release of dbt-docs in v0.11.0. Starting in dbt v0.19.0, we are committing to a stable and sustainable way of versioning, documenting, and validating dbt artifacts.
When are artifacts produced?
Most dbt commands (and corresponding RPC methods) produce artifacts:
- manifest: produced by
compile
,run
,test
,docs generate
,ls
- run results: produced by
run
,test
,seed
,snapshot
,docs generate
- catalog: produced by
docs generate
- sources: produced by
source snapshot-freshness
Common metadata
▶Changelog
All artifacts produced by dbt include a metadata
dictionary with these properties:
dbt_version
: Version of dbt that produced this artifact.dbt_schema_version
: URL of this artifact's schema. See notes below.generated_at
: Timestamp in UTC when this artifact was produced.adapter_type
: The adapter (database), e.g.postgres
,spark
, etc.env
: Any environment variables prefixed withDBT_ENV_CUSTOM_ENV_
will be included in a dictionary, with the prefix-stripped variable name as its key.invocation_id
: Unique identifier for this dbt invocation
In the manifest, the metadata
may also include:
send_anonymous_usage_stats
: Whether this invocation sent anonymous usage statistics while executing.project_id
: Project identifier, hashed fromproject_name
, sent with anonymous usage stats if enabled.user_id
: User identifier, stored by default in~/dbt/.user.yml
, sent with anonymous usage stats if enabled.
Notes:
- The structure of dbt artifacts is canonized by JSON schemas, which are hosted at schemas.getdbt.com.
- As of v0.19.0, all artifacts are
v1
: - The previous
v0
schemas are also available for ease of comparison. These represent the artifact schemas as of v0.18.1. In prior versions, artifacts were not standardized or versioned. - Going forward, artifact versions may change in any minor version of dbt (
v0.x.0
). Artifact version updates are not guaranteed to align with each other—for instance, dbt may introducemanifest/v2
while still usingsources/v1
. We will always include artifact schema updates as breaking changes in the release notes.