[英]Using arguments in bigquery schema for Google DataFusion
I am trying to use runtime arguments in bigquery schema defintion in Bigquery sink plugin.我正在尝试在 Bigquery 接收器插件的 bigquery 模式定义中使用运行时 arguments。 It is just two columns.
它只是两列。 Definition in argument setter.json.
参数 setter.json 中的定义。 -
-
{
"arguments" : [
{"name":"bq.config.table","value":"activity_category"},
{
"name" : "sqloutput_schema",
"type" : "schema",
"value" :
[
{"name":"activity_category_id","type":"string","nullable":true},
{"name":"activity_category_description","type":"string"}
]
}
]
}
Issue is in the 'sqloutput_schema', which is failing during runtime - PFA screenshot of plugin:-问题出在“sqloutput_schema”中,它在运行时失败 - 插件的 PFA 屏幕截图:-
Error received - Spark program 'phase-2' failed with error: Argument 'sqloutput_schema' is not defined.Please check the system logs for more details. io.cdap.cdap.api.macro.InvalidMacroException: Argument 'sqloutput_schema' is not defined.
收到错误 -
Spark program 'phase-2' failed with error: Argument 'sqloutput_schema' is not defined.Please check the system logs for more details. io.cdap.cdap.api.macro.InvalidMacroException: Argument 'sqloutput_schema' is not defined.
Spark program 'phase-2' failed with error: Argument 'sqloutput_schema' is not defined.Please check the system logs for more details. io.cdap.cdap.api.macro.InvalidMacroException: Argument 'sqloutput_schema' is not defined.
I am unable to find a solution as to why this is failing.我找不到解决失败原因的解决方案。
The problem is your schema definition.问题是您的模式定义。 I had the same use-case and my argument was of type string and the value had the following format -
我有相同的用例,我的参数是字符串类型,值具有以下格式 -
"{\"name\":\"etlSchemaBody\",\"type\":\"record\",\"fields\":
[
{\"name\":\"Id\",\"type\":\"int\"},
{\"name\":\"name\",\"type\":\"string\"}
]}"
So change the type of the argument of the schema and the schema json following the format above.因此,按照上面的格式更改模式的参数类型和模式 json。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.