Skip to main content

Source properties

Related documentation#

Overview#

Source properties can be declared in .yml files in your models/ directory (as defined by the source-paths config).

You can name these files whatever_you_want.yml, and nest them arbitrarily deeply in subfolders within the models/ directory.

models/<filename>.yml
version: 2
sources:  - name: <string> # required    description: <markdown_string>    database: <database_name>    schema: <schema_name>    loader: <string>    loaded_at_field: <column_name>    meta: {<dictionary>}    tags: [<string>]
    overrides: <string>
    freshness:      warn_after:        count: <positive_integer>        period: minute | hour | day      error_after:        count: <positive_integer>        period: minute | hour | day      filter: <where-condition>
    quoting:      database: true | false      schema: true | false      identifier: true | false
    tables:      - name: <string> #required        description: <markdown_string>        meta: {<dictionary>}        identifier: <table_name>        loaded_at_field: <column_name>        tests:          - <test>          - ... # declare additional tests        tags: [<string>]        freshness:          warn_after:            count: <positive_integer>            period: minute | hour | day          error_after:            count: <positive_integer>            period: minute | hour | day          filter: <where-condition>
        quoting:          database: true | false          schema: true | false          identifier: true | false        external: {<dictionary>}        columns:          - name: <column_name> # required            description: <markdown_string>            meta: {<dictionary>}            quote: true | false            tests:              - <test>              - ... # declare additional tests            tags: [<string>]          - name: ... # declare properties of additional columns
      - name: ... # declare properties of additional source tables
  - name: ... # declare properties of additional sources

Example#

models/<filename>.yml
version: 2
sources:  - name: jaffle_shop    database: raw    schema: public    loader: emr # informational only (free text)    loaded_at_field: _loaded_at # configure for all sources
    # meta fields are rendered in auto-generated documentation    meta:      contains_pii: true      owner: "@alice"
    # Add tags to this source    tags:      - ecom      - pii
    quoting:      database: false      schema: false      identifier: false
    tables:      - name: orders        identifier: Orders_        loaded_at_field: updated_at # override source defaults        columns:          - name: id            tests:              - unique
          - name: price_in_usd            tests:              - not_null
      - name: customers        quoting:          identifier: true # override source defaults        columns:            tests:              - unique