GCP Dataflow Quickstart error running WordCount example: Unknown lifecycle phase “.mainClass=org.apache.beam.examples.WordCount”

Question

I am trying to go through the "Quickstart Using Java" for GCP Cloud Dataflow from here https://cloud.google.com/dataflow/docs/quickstarts/quickstart-java-maven and am getting an error in the "Run WordCount locally" step.

Running this at a PowerShell prompt

PS C:\apache-maven-3.6.2\word-count-beam> mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="--output=.\output\"

I receive a BUILD FAILURE message with:

[ERROR] Unknown lifecycle phase ".mainClass=org.apache.beam.examples.WordCount". You must specify a valid lifecycle phase or a goal in the format: or:[:]:. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site-deploy. -> [Help 1]

When I add -e I also get the stack trace:

at org.apache.maven.lifecycle.internal.DefaultLifecycleExecutionPlanCalculator.calculateLifecycleMappings (DefaultLifecycleExecutionPlanCalculator.java:248) at org.apache.maven.lifecycle.internal.DefaultLifecycleExecutionPlanCalculator.calculateMojoExecutions (DefaultLifecycleExecutionPlanCalculator.java:217) at org.apache.maven.lifecycle.internal.DefaultLifecycleExecutionPlanCalculator.calculateExecutionPlan (DefaultLifecycleExecutionPlanCalculator.java:126) at org.apache.maven.lifecycle.internal.DefaultLifecycleExecutionPlanCalculator.calculateExecutionPlan (DefaultLifecycleExecutionPlanCalculator.java:144) at org.ZB6EFD606D118D0F62066E31419F F04CCZ.maven.lifecycle.internal.builder.BuilderCommon.resolveBuildPlan (BuilderCommon.java:97) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:111) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.Z402C5D9AF6B43711EA070BEE 5170F74DZ.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288) at org.apache.maven.cli.MavenCli.main (MavenCli.java:192) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.Z93F725A0742 3FE1C889F448B33D21F46Z:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:567) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

I am using jdk-13.0.1 and maven 3.6.2 and have setup JAVA_HOME and added JAVA_HOME\Bin and Maven to my PATH.

Any idea what is going wrong?

Answer 1

I see that you are using JDK-13.0.1 whereas Apache Beam currently supports Java 8 . Below are the pre-requisites for Java and Maven.

1. Download and install the Java Development Kit (JDK) version 8. Verify that the JAVA_HOME environment variable is set and points to your JDK installation.
2. Download and install Apache Maven by following Maven's installation guide for your specific operating system.

Once you have done the installs for Java and Maven, you can get the code using the following command

mvn archetype:generate \
      -DarchetypeGroupId=org.apache.beam \
      -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
      -DarchetypeVersion=2.16.0 \
      -DgroupId=org.example \
      -DartifactId=word-count-beam \
      -Dversion="0.1" \
      -Dpackage=org.apache.beam.examples \
      -DinteractiveMode=false

To run the code locally

cd word-count-beam/

mvn compile exec:java \
      -Dexec.mainClass=org.apache.beam.examples.WordCount \
      -Dexec.args="--output=./output/"

Answer 2

If you are using powershell, then instead of "-Dexec.mainClass" use "-D exec.mainClass"