[英]How to query json with python UDF or json_extract in BigQuery?
[英]how to read multiple levels of JSON data in Big Query using JSON_EXTRACT or JSON_EXTRACT_SCALAR
我正在尝试使用 Bigquery 中的 JSON_EXTRACT 读取 Bigquery 中的以下 JSON 结构..它说不受支持的运算符路径“*”
在 BigQuery 中尝试了所有可能的方法并请求您的帮助
错误: JSONPath 中不支持的运算符:*
****JSON 数据:** 只是具有多个值并且在阅读时有问题的部分。 需要读取下面的所有 4 个“id”值作为一个例子,并且需要读取所有其他列以及组合部分下的产生 4 行不同 ID、类型等**
"Combos": [
{
"Id": "1111",
"Type": 0,
"Description": "ABCD",
"ComboDuration": {
"StartDate": "2009-10-26T08:00:00",
"EndDate": "2009-10-29T08:00:00"
}
},
{
"Id": "2222",
"Type": 1,
"Description": "XYZ",
"ComboDuration": {
"StartDate": "2019-10-26T08:00:00",
"EndDate": "2019-10-29T08:00:00"
}
},
{
"Id": "39933",
"Type": 3,
"Description": "General",
"ComboDuration": {
"StartDate": "2019-10-26T08:00:00",
"EndDate": "2019-10-29T08:00:00"
}
},
{
"Id": "39934",
"Type": 2,
"Description": "ABCDXYZ",
"ComboDuration": {
"StartDate": "2019-10-26T08:00:00",
"EndDate": "2019-10-29T08:00:00"
}
},
]
****Code:** P.S - conv_column is a string column where my JSON structure stored**
SELECT
JSON_EXTRACT(conv_column,"$.Combos.*.Id") as combo_id
from lz.json_file
SELECT JSON_EXTRACT(conv_column,"$.Combos[*].Id") as combo_id
from lz.json_file
SELECT JSON_EXTRACT(conv_column,"$.Combos[?@.Id]") as combo_id
from lz.json_file
下面是标准 SQL 的 BigQuery 示例
#standardSQL
CREATE TEMP FUNCTION jsonparse(input STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS """
return JSON.parse(input).map(x=>JSON.stringify(x));
""";
WITH `project.lz.json_file` AS (
SELECT '''{
"Combos": [ {
"Id": "1111",
"Type": 0,
"Description": "ABCD",
"ComboDuration": {
"StartDate": "2009-10-26T08:00:00",
"EndDate": "2009-10-29T08:00:00"
} }, {
"Id": "2222",
"Type": 1,
"Description": "XYZ",
"ComboDuration": {
"StartDate": "2019-10-26T08:00:00",
"EndDate": "2019-10-29T08:00:00"
} }, {
"Id": "39933",
"Type": 3,
"Description": "General",
"ComboDuration": {
"StartDate": "2019-10-26T08:00:00",
"EndDate": "2019-10-29T08:00:00"
} }, {
"Id": "39934",
"Type": 2,
"Description": "ABCDXYZ",
"ComboDuration": {
"StartDate": "2019-10-26T08:00:00",
"EndDate": "2019-10-29T08:00:00"
} }]} ''' AS conv_column
)
SELECT
JSON_EXTRACT_SCALAR(combo, '$.Id') AS Id,
JSON_EXTRACT_SCALAR(combo, '$.Type') AS Type,
JSON_EXTRACT_SCALAR(combo, '$.Description') AS Description,
JSON_EXTRACT_SCALAR(combo, '$.ComboDuration.StartDate') AS StartDate,
JSON_EXTRACT_SCALAR(combo, '$.ComboDuration.EndDate') AS EndDate
FROM `project.lz.json_file`,
UNNEST(jsonparse(JSON_EXTRACT(conv_column, '$.Combos'))) combo
带输出
Row Id Type Description StartDate EndDate
1 1111 0 ABCD 2009-10-26T08:00:00 2009-10-29T08:00:00
2 2222 1 XYZ 2019-10-26T08:00:00 2019-10-29T08:00:00
3 39933 3 General 2019-10-26T08:00:00 2019-10-29T08:00:00
4 39934 2 ABCDXYZ 2019-10-26T08:00:00 2019-10-29T08:00:00
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.