Extract nested values as columns Google BigQuery?

Question

I have a table with nested values, like the following:

I'd like to grab the values, with keys as columns without multiple cross joins. ie

SELECT 
owner_id, 
owner_type, 
domain, 
metafields.value AS name, 
metafields.value AS image, 
metafields.value AS location, 
metafields.value AS draw
FROM 
example_table

Obviously, the above won't work for this, but the following output would be desired:

In the actual table there are hundreds of metafields per owner_id, and hundreds of owner_ids, and owner_types. Multiple joins to other tables for owner_types is fine, but for the same owner type, I don't want to have to join multiple times.

Basically, I need to be able to select the key to which the column corresponds, and display the relevant value for that column. Without, having to display every metafield available.

Any way of doing this?

Answer 1

You can use the subqueries and SAFE_offset statement and get a value from an array at a specific location. Also, you need to use STRING_AGG , which returns a value (either STRING or BYTES) obtained by concatenating non-null values.

With the information you shared, you can use the query below.

With this code, you will get all the columns separated by a comma:

WITH sequences AS
 (
     SELECT 1 as ID,"product" AS owner_type,"beta.com" AS domain,["name","image","lcation","draw"] AS metalfields_key, ["big","pic.png","utha","1"] AS metalfields_value
     ),
 Val as(
 SELECT distinct id, owner_type,domain, value FROM sequences, sequences.metalfields_value as value, sequences.metalfields_key
 ), text as(
 SELECT
 id, owner_type, domain,
 STRING_AGG(value ORDER BY value) AS Text
FROM Val 
GROUP BY owner_type, domain, id
 )
 

In this code, you will get each element that is separated by a comma and return them by columns.

SELECT DISTINCT t1.id, t1.owner_type,domain,
split(t1.text, ',')[SAFE_offset(1)] as name,
split(t1.text, ',')[SAFE_offset(2)] as image,
split(t1.text, ',')[SAFE_offset(3)] as location,
split(t1.text, ',')[SAFE_offset(0)] as draw
from text as t1

You can see the result.

Answer 2

Consider below approach

select * except(id) from (
  select t.* except(metafields), 
    to_json_string(t) id, key, value
  from your_table t, unnest(metafields) kv
)
pivot (min(value) for key in ('name', 'image', 'location', 'draw'))          

if applied to sample data in your question - output is