[英]Adding support for JS UDF in Google Dataflow template
I have this code from Google Cloud Platform Dataflow Templates. 我有来自Google Cloud Platform数据流模板的代码 。
I wish to add more functionalities to it, namely, I wish to add support for JavaScript UDF. 我希望为其添加更多功能,即希望添加对JavaScript UDF的支持。 When I try to compile the file, using this:
当我尝试编译文件时,使用以下命令:
mvn compile exec:java \
-Dexec.mainClass=com.google.cloud.teleport.templates.${PIPELINE_NAME} \
-Dexec.cleanupDaemonThreads=false \
-Dexec.args=" \
--project=${PROJECT_ID} \
--stagingLocation=gs://${PROJECT_ID}/dataflow/${PIPELINE_FOLDER}/staging \
--tempLocation=gs://${PROJECT_ID}/dataflow/${PIPELINE_FOLDER}/temp \
--runner=DataflowRunner \
--windowDuration=2m \
--numShards=1 \
--topic=projects/${PROJECT_ID}/topics/windowed-files \
--outputDirectory=gs://${PROJECT_ID}/temp/ \
--outputFilenamePrefix=windowed-file \
--outputFilenameSuffix=.txt"
When compiling the file, I get the following error: 编译文件时,出现以下错误:
An exception occured while executing the Java class. Class interface com.google.cloud.teleport.templates.PubsubToText$Options missing a property named 'topic'. -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. Class interface com.google.cloud.teleport.templates.PubsubToText$Options missing a property named 'topic'.
Even though, I've passed the --topic
flag with appropriate values plugged in. 即使,我通过了
--topic
标志,并插入了适当的值。
The example at the top is wrong. 顶部的示例是错误的。 You have to pass
--inputTopic
instead of --topic
. 您必须传递
--inputTopic
而不是--topic
。 You can see this in the code where the ValueProvider
is defined: 您可以在定义
ValueProvider
的代码中看到以下内容:
@Description("The Cloud Pub/Sub topic to read from.")
@Required
ValueProvider<String> getInputTopic();
void setInputTopic(ValueProvider<String> value);
You can also run the template from the Console UI and the job details will show that the option is indeed inputTopic
: 您也可以从控制台 UI运行模板,作业详细信息将显示该选项确实为
inputTopic
:
现在,javadoc中的调用示例应该在此处反映正确的输入参数(--inputTopic)。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.