Skip to main content

Rockset setup

Vendor-supported plugin

Certain core functionality may vary. If you would like to report a bug, request a feature, or contribute, you can check out the linked repository and open an issue.

  • Maintained by: Rockset, Inc.
  • Authors: Rockset, Inc.
  • GitHub repo: rockset/dbt-rockset
  • PyPI package: dbt-rockset
  • Slack channel: #dbt-rockset
  • Supported dbt Core version: v0.19.2 and newer
  • dbt support: Not Supported
  • Minimum data platform version: ?

Installing dbt-rockset

Use pip to install the adapter. Before 1.8, installing the adapter would automatically install dbt-core and any additional dependencies. Beginning in 1.8, installing an adapter does not automatically install dbt-core. This is because adapters and dbt Core versions have been decoupled from each other so we no longer want to overwrite existing dbt-core installations. Use the following command for installation:

python -m pip install dbt-core dbt-rockset

Configuring dbt-rockset

For Rockset-specific configuration, please refer to Rockset configs.

Connecting to Rockset with dbt-rockset​

The dbt profile for Rockset is very simple and contains the following fields:

profiles.yml
rockset:
  target: dev
  outputs:
    dev:
      type: rockset
      workspace: [schema]
      api_key: [api_key]
      api_server: [api_server] # (Default is api.rs2.usw2.rockset.com)

Materializations​

TypeSupported?Details
viewYESCreates a view.
tableYESCreates a collection.
ephemeralYESExecutes queries using CTEs.
incrementalYESCreates a collection if it doesn't exist, and then writes results to it.

Caveats​

  1. unique_key is not supported with incremental, unless it is set to _id, which acts as a natural unique_key in Rockset anyway.
  2. The table materialization is slower in Rockset than most due to Rockset's architecture as a low-latency, real-time database. Creating new collections requires provisioning hot storage to index and serve fresh data, which takes about a minute.
  3. Rockset queries have a two-minute timeout. Any model which runs a query that takes longer to execute than two minutes will fail.

Was this page helpful?

0
Edit this page