如何解决“无法执行目标 org.codehaus.mojo:exec-maven-plugin:1.6.0:java”错误从 GCP 中的 VM 实例运行 Dataflow 模板？

Question

我正在尝试在 VM 实例（操作系统：“linux”，版本：“4.9.0-11-amd64”，分销商：Debian）上执行名为PubSubToBigQuery.java的数据流模板到 GNU/Linux 9.11 输入来自 Pub/Sub 订阅的消息并将它们写入 BigQuery 表（暂时不修改模板）。 为此，我将GitHub DataflowTemplates存储库克隆到了 $HOME/opt/ 目录中的 Cloud Shell 中。 按照 README 文档，我安装了 Java 8 和 Maven 3：

• Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) Maven home: /opt/maven Java version : 1.8.0_232, vendor: Oracle Corporation Java home: /usr/lib/jvm/java -8-openjdk-amd64/jre 默认语言环境：en_US，平台编码：UTF-8 操作系统名称：“linux”，版本：“4.9.0-11-amd64”，arch：“amd64”，家族：“unix”

在构建整个项目之后，这就是我试图从命令行执行以编译代码：

sudo mvn -X compile exec:java -Dexec.mainClass=com.google.cloud.teleport.templates.PubSubToBigQuery - 
Dexec.cleanupDaemonThreads=false -Dexec.args=" --project=real-demand -- 
stagingLocation=gs://rd_pubsub/staging --streaming=true"

这是我得到的错误：

[WARNING]
java.lang.ClassNotFoundException: com.google.cloud.teleport.templates.PubSubToBigQuery.java
    at java.net.URLClassLoader.findClass (URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:357)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:270)
    at java.lang.Thread.run (Thread.java:748)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:21 min
[INFO] Finished at: 2019-11-26T10:47:57Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. com.google.cloud.teleport.templates.PubSubToBigQuery.java -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project google-cloud-teleport-java: An exception occured while executing the Java class. com.google.cloud.teleport.templates.PubSubToBigQuery.java
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception occured while executing the Java class. com.google.cloud.teleport.templates.PubSubToBigQuery.java
    at org.codehaus.mojo.exec.ExecJavaMojo.execute (ExecJavaMojo.java:339)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.ClassNotFoundException: com.google.cloud.teleport.templates.PubSubToBigQuery.java
    at java.net.URLClassLoader.findClass (URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:357)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:270)
    at java.lang.Thread.run (Thread.java:748)
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

在这篇文章或这个讨论讨论之后，我还在 pom.xml 中添加了这段 XML：

<properties>
<testNgMainClass>defaultMainClass</testNgMainClass>
<args>defaultArgs</args>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<configuration>
<classpathScope>test</classpathScope>
<mainClass>${testNgMainClass}</mainClass>
<arguments>${args}</arguments>
</configuration>
<executions>
<execution>
<id>run-selenium</id>
<phase>integration-test</phase>
<goals><goal>java</goal></goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

但它仍然无法正常工作。

Answer 1

我可以看到您没有指定执行模板所需的所有参数，并且正如您所提到的，您没有修改文件。

• inputTopic=projects/${PROJECT_ID}/topics/input-topic-name
• outputTableSpec=${PROJECT_ID}:dataset-id.output-table
• outputDeadletterTable=${PROJECT_ID}:dataset-id.deadletter-table

该命令应如下所示：

mvn compile exec:java -Dexec.mainClass=com.google.cloud.teleport.templates.PubSubToBigQuery -Dexec.cleanupDaemonThreads=false \
 -Dexec.args=" --project=PROJECT_ID \
 --stagingLocation=gs://bucket-name/dataflow \
 --streaming=true \
 --inputTopic=projects/PROJECT_ID/topics/input-topic-name \
 --outputTableSpec=PROJECT_ID:dataset-id.deadletter-table \
 --outputDeadletterTable=PROJECT_ID:dataset-id.deadletter-table"

另一方面，我建议将模板部署到 GCP 并使用 gcloud 命令执行，按照 PubSubToBigQuery.java 文件中给出的说明进行操作

Answer 2

要克隆、编译和运行数据流模板，必须在 GCP 项目中启用所有必要的 API：

• 数据流
• 计算引擎
• Stackdriver 日志记录
• 云储存
• 云存储 JSON
• 大查询
• 发布订阅

为此，您可以单击此帮助链接：

https://console.cloud.google.com/flows/enableapi?apiid=dataflow,compute_component,logging,storage_component,storage_api,bigquery,pubsub