Skip to main content

Merge jobs in dbt Cloud

You can set up a merge job to implement a continuous development (CD) workflow in dbt Cloud. The merge job triggers a dbt job to run when someone merges Git pull requests into production. This creates a seamless development experience where changes made in code will automatically update production data.

By using CD in dbt Cloud, you can take advantage of deferral to build only the edited model and any downstream changes. With merge jobs, state will be updated almost instantly, always giving the most up-to-date state information in dbt Explorer.

Prerequisites​

  • You have a dbt Cloud account.
  • You have set up a connection with your Git provider. This integration lets dbt Cloud run jobs on your behalf for job triggering.
    • If you're using a native GitLab integration, you need a paid or self-hosted account that includes support for GitLab webhooks and project access tokens. If you're using GitLab Free, merge requests will trigger CI jobs but CI job status updates (success or failure of the job) will not be reported back to GitLab.
  • You are set up to receive push notifications from your Git provider. For details, refer to Job notifications.

Set up job trigger on Git merge​

  1. On your deployment environment page, click Create Job > Merge Job.
  2. Options in the Job settings section:
    • Job name — Specify the name for the merge job.
    • Description — Provide a descripion about the job.
    • Environment — By default, it’s set to the environment you created the job from.
  3. In the Git trigger section, the Run on merge option is enabled by default. Every time a PR merges (to the main branch) in your Git repo, this job will get triggered to run.
  4. Options in the Execution settings section:
    • Commands — By default, it includes the dbt build --select state:modified+ command. This informs dbt Cloud to build only new or changed models and their downstream dependents. Importantly, state comparison can only happen when there is a deferred environment selected to compare state to. Click Add command to add more commands that you want to be invoked when this job runs.
    • Compare changes against an environment (Deferral) — By default, it's set to the environment you created the job from. This option allows dbt Cloud to check the state of the code in the PR against the code running in the deferred environment, so as to only check the modified code, instead of building the full table or the entire DAG.
  5. (optional) Options in the Advanced settings section:
    • Environment variables — Define environment variables to customize the behavior of your project when this job runs.
    • Target name — Define the target name. Similar to Environment Variables, this option lets you customize the behavior of the project.
    • Run timeout — Cancel this job if the run time exceeds the timeout value.
    • dbt version — By default, it’s set to inherit the dbt version from the environment. dbt Labs strongly recommends that you don't change the default setting. This option to change the version at the job level is useful only when you upgrade a project to the next dbt version; otherwise, mismatched versions between the environment and job can lead to confusing behavior.
    • Threads — By default, it’s set to 4 threads. Increase the thread count to increase model execution concurrency.
    • Run source freshness — Enable this option to invoke the dbt source freshness command before running this job. Refer to Source freshness for more details.

Example​

Example of creating a merge jobExample of creating a merge job
0
Edit this page