dbt Jinja 宏将列集传递给列表并执行计算和重命名

Question

I have a code below where:

1. I am fetching all the relevant columns from two tables and renaming them to a similar keyword,

2. bringing all the data into 1 row.

3. running a macro to identify mismatch

I would like to see the final names of each column as s1_xxx or s2_xxx . s1_ indicates the column is from table 1 and s2_xxx indicates it is from table2

The problem is that I have to pass the columns as a tuple/list as where format('%t', (s1_id,s1_qty,s1_price)),= format('%t', (s2_id,s2_qty,s2_price)) . But there could be multiple columns, in my example I am just showing 3. So the code could go like:

(s1_id,s1_qty,s1_price, s1_aa, s1_ab, s1_ac.......n) . In that case, it doesn't look that readable

with source1 as (
select 
b.id as s1_id,
b.qty as s1_qty,
a.price as s1_price
from <table> as a
,unnest <details> as b
 where b.status != 'canceled'
),

source2 as (
select 
un_id  as s2_id, 
acqty_ as s2_qty, 
price_per_unit as s2_price  
from <table2>
where city != 'delhi'
) ,

compare as (
select *
from source1 s1
full outer join source2 s2
on id = id_
where format('%t', (s1_id,s1_qty,s1_price)) != format('%t', (s2_id,s2_qty,s2_price))
)

Select 

*,   {{ create_calc(s1_price,s2_price) }} ,   {{ create_calc(s1_qty,s2_qty) }}

from compare
## this would then generate two columns:

**is_price_mismatch and is_qty_mismatch** based on the macro (see below)

{{ create_calc(col1,col2) }} indicates the following:

{% macro create_calc(col1, col2) -%}
if(ifnull({{ col1 }} != {{ col2 }},true) is true,1,0) as {{ col1 | replace(‘s1_,'is_') }}_mismatch
{% endmacro %}

I wanted to check if it is possible to use jinja to kind of define the columns in a list/tuple and then pass it in the where clause (see below):

where format('%t', (s1_id,s1_qty,s1_price)) would convert to something like:

where format('%t', list) but even if I do this I will have to create two lists? as the names of the columns have different prefixes s1_xx, s2_xx:

1. col_list = (s1_id, s1_qty, s1_price,.......n)
2. col_list2 = (s2_id, s2_qty, s2_price,.......n)

And then passing it to where clause:

where format('%t', (col_list)),= format('%t', (col_list2))

or is there a way I can pass one list and rename the columns along with it via some code?

col_list = (id, qty, price) ## note the the

final output would then be:

• s1_id,
• s1_qty,
• s1_price,
• s2_id,
• s2_qty,
• s2_price
• is_price_mismatch
• is_qty_mismatch

I was hoping someone can help me how I can code this logic or maybe any other solution that would work with my existing code above / minor adjustments

Update:

I tried the following logic:

{% set cols = ("price", "qty") %}

and passed it to my format() :

format('%t',{{ cols }}) as serial

but the results are not correct somehow the ' is being attached to the beginning and ending of the column names: (can be seen in the compiled version:

format('%t',('price', 'quantity')) as serial

what i need is:

format('%t',(price, quantity)) as serial

Answer 1

it can be solved using two ways:

{% set cols = ('id' , 'qty') %}


## and then using 

format('%t',{{ cols_ | replace('\'','') }})

or

{% set cols = 'id,quantity' %}


## and then using : 

format('%t',({{ cols_ }})) as d365_cols