SQL parse

dbt uses the Python sqlparse library when it parses SQL. For large or complex queries, you may need to adjust how sqlparse groups tokens to avoid performance issues or parsing limits.

You can use the --sqlparse flag to adjust how sqlparse groups tokens. This maps to the grouping limits described in the sqlparse Security and Performance Considerations.

Supported keys:

Key	What it does
MAX_GROUPING_DEPTH	Sets the maximum nesting depth when grouping tokens
MAX_GROUPING_TOKENS	Sets the maximum number of tokens that can be grouped at once

Each key accepts a single value: an integer or null. The default value for both keys is null, which means dbt does not apply a custom limit.

sqlparse enforces built-in limits (MAX_GROUPING_DEPTH = 100 and MAX_GROUPING_TOKENS = 10000) to prevent excessive resource usage. Setting a value to null disables the limit entirely.

For example, the following command sets both grouping limits:

dbt compile --sqlparse '{"MAX_GROUPING_DEPTH": 200, "MAX_GROUPING_TOKENS": 20000}'

You can configure only one key to set a single limit. Any unspecified key defaults to null. In the following example, only the depth limit is raised to 500; MAX_GROUPING_TOKENS is not set, so sqlparse does not apply a token limit for this run.

dbt compile --sqlparse '{"MAX_GROUPING_DEPTH": 500}'

If no keys are provided, both limits remain unset (null), and sqlparse does not apply any grouping limits.

You can use --sqlparse with the following commands:

• dbt compile
• dbt run
• dbt build
• dbt test
• dbt seed
• dbt snapshot
• dbt source freshness
• dbt docs generate
• dbt show

Was this page helpful?

Privacy policyCreate a GitHub issue

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.