我想用 BigQuery 提取 Json 格式的数据。 UDF 或 json_extract

Question

I have a table with the following structure.

user_id int,

purchase_ids string（in Json format）

The JSON contained in one record in this table looks like this:

user_id = 0001

1:{
  shop_id:1,
  product_id :1111,
  value: 1
},
2:{
  shop_id:1,
  product_id :2222,
  value: 1
},
3:{
  shop_id:1,
  product_id :3333,
  value: 1
},
.... Numbers fluctuate as records approach

Final output to aim for

| user_id | shop_id | product_id | value |
| 0001    | 1       |  1111      | 1     |
| 0001    | 1       |  2222      | 1     |
| 0001    | 1       |  3333      | 1     |

I tried the following query when I was thinking but it doesn't seem to be done right shop_id and product_id return null.

CREATE TEMP FUNCTION jsonparse(json_row STRING)
  RETURNS STRING
  LANGUAGE js AS """
  var res = array();
  json_row.forEach(([key, value]) => {
     res = value;
  });
  return res
""";

with
parse as(
select
  user_id,
  jsonparse(purchase_ids) as pids
from
  sample
)


select
  user_id,
  JSON_EXTRAXT(pid,"$.shop_id") as shop_id,
  JSON_EXTRAXT(pid,"$.product_id") as product_id
from
  parse,
  unnest(pids,",") pid

How do you get it right in this situation?

Answer 1

From my point of view, your use case needs to use a NESTED and REAPEATED column that can be represented with a json structure. For example, the following query return the result you are looking for:

WITH users AS
  (SELECT "0001" as user_id, ARRAY<STRUCT<shop_id INT64, product_id INT64, value INT64>>[(1, 1111,1),
    (1, 2222,1), (1, 3333,1)] AS shops)
SELECT u.user_id, s.*
FROM users u, UNNEST(shops) s;

For simplicity you can create this type of column from the Console to try this approach by following this guide .

Answer 2

Below is the working version of your use case (BigQuery Standard SQL)

#standardSQL
CREATE TEMP FUNCTION jsonparse(input STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS """
  return JSON.parse(input).map(x=>JSON.stringify(x));
"""; 
WITH sample AS (
  SELECT "0001" AS user_id, 
  '''[{"shop_id": 1, "product_id" :1111, "value": 1},
  {"shop_id": 1, "product_id" :2222, "value": 1},
  {"shop_id": 1, "product_id" :3333, "value": 1}]''' AS purchase_ids
), parse AS (
  SELECT user_id,
    jsonparse(purchase_ids) AS pids
  FROM sample
) 
SELECT
  user_id,
  JSON_EXTRACT(pid,"$.shop_id") AS shop_id,
  JSON_EXTRACT(pid,"$.product_id") AS product_id,
  JSON_EXTRACT(pid,"$.value") AS value
FROM parse,
UNNEST(pids) pid

with result

Row user_id shop_id product_id  value    
1   0001    1       1111        1    
2   0001    1       2222        1    
3   0001    1       3333        1