Skip to main content

Rockset setup

Vendor-supported plugin

Certain core functionality may vary. If you would like to report a bug, request a feature, or contribute, you can check out the linked repository and open an issue.

  • Maintained by: Rockset, Inc.
  • Authors: Rockset, Inc.
  • GitHub repo: rockset/dbt-rockset
  • PyPI package: dbt-rockset
  • Slack channel: #dbt-rockset
  • Supported dbt Core version: v0.19.2 and newer
  • dbt support: Not Supported
  • Minimum data platform version: ?

Installing dbt-rockset

Use pip to install the adapter. Before 1.8, installing the adapter would automatically install dbt-core and any additional dependencies. Beginning in 1.8, installing an adapter does not automatically install dbt-core. This is because adapters and dbt Core versions have been decoupled from each other so we no longer want to overwrite existing dbt-core installations. Use the following command for installation:

python -m pip install dbt-core dbt-rockset

Configuring dbt-rockset

For Rockset-specific configuration, please refer to Rockset configs.

Connecting to Rockset with dbt-rockset​

The dbt profile for Rockset is very simple and contains the following fields:

profiles.yml
rockset:
  target: dev
  outputs:
    dev:
      type: rockset
      workspace: [schema]
      api_key: [api_key]
      api_server: [api_server] # (Default is api.rs2.usw2.rockset.com)

Materializations​

TypeSupported?Details
viewYESCreates a view.
tableYESCreates a collection.
ephemeralYESExecutes queries using CTEs.
incrementalYESCreates a collection if it doesn't exist, and then writes results to it.

Caveats​

  1. unique_key is not supported with incremental, unless it is set to _id, which acts as a natural unique_key in Rockset anyway.
  2. The table materialization is slower in Rockset than most due to Rockset's architecture as a low-latency, real-time database. Creating new collections requires provisioning hot storage to index and serve fresh data, which takes about a minute.
  3. Rockset queries have a two-minute timeout. Any model which runs a query that takes longer to execute than two minutes will fail.

Was this page helpful?

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

0
Edit this page