使用 DBT 在 bigquery 中更新表

Question

I am trying to update a table in bigquery using DBT. The below command executes in bigquery;

Update {{ ref('my_table') }}
SET variable = 'value'
WHERE lower(variable) LIKE '%XX%' or lower(variable) like '%YY%'

However when I run it in DBT I get the following error

Server error: Database Error in rpc request (from remote system)
Syntax error: Expected end of input but got keyword LIMIT at [4:1]

Does anyone know why this is happening and how to resolve?

Answer 1

It's a little unintuitive at first I know but with dbt, every model is a select statement .

You should instead think of doing something like:

with cte as (
     select * from {{ ref('my_table') }}
     where <criteria>
)
 select col1,
        col2,
        'value' as col3
 from cte

Or possibly even simpler:

SELECT 
  'value' as variable
FROM {{ ref('my_table') }}
WHERE lower(variable) LIKE '%XX%' or lower(variable) like '%YY%'

Simply because during the dbt run cycle, the new values will be materialized into the new model.

However, if you are looking for ways to clean underlying tables in a DRY way, I'd highly recommend this thread Modeling SQL Update Statements from the dbt discourse for some patterns on managing statements which handle specific value cleaning. Example from Kyle Ries:

{% set mappings = {'something': 'boo', 'something-else': 'boo-else'} %}

with source as (
        select * from {{ ref(‘stg_foobar’) }}
),

final as ( 

        select
            case
              {% for old, new in mappings %}
                when other_column like ‘{{old}}’ then ‘{{new}}’
              {% endfor %}
            end as column_name
        from
            source

)

select * from final