在Eclipse上使用Dataflow Runner的Apache Beam MinimalWordcount示例

Question

I am trying to run the MinimalWordCount example using the DataFlowRunner from Eclipse on windows using MinimalWordCount -->Run As Java Application from with in eclipse , its the same stock code from the example using my gcs bucket , however I consistently get the following exception , can some one let me know whats the issue here?

1. I have verified that the bucket name is correct.
2. I already ran the gcloud init on my Windows machine.

Exception in thread "main" java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions) at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:233) at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162) at org.apache.beam.sdk.runners.PipelineRunner.fromOptions(PipelineRunner.java:56) at org.apache.beam.sdk.Pipeline.create(Pipeline.java:135) at com.dynaobject.apachebeamexperiment.MinimalWordCount.main(MinimalWordCount.java:77) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:222) ... 4 more Caused by: java.lang.IllegalA rgumentException: DataflowRunner requires gcpTempLocation, but failed to retrieve a value from PipelineOptions at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:212) ... 9 more Caused by: java.lang.IllegalArgumentException: Error constructing default value for gcpTempLocation: tempLocation is not a valid GCS path, gs://tempxyz. at org.apache.beam.sdk.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:219) at org.apache.beam.sdk.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:205) at org.apache.beam.sdk.options.ProxyInvocationHandler.returnDefaultHelper(ProxyInvocationHandler.java:575) at org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:516) at org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:155) at com.sun.proxy.$Proxy15.getGcpTempLocation(Unknown Source) at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:210) ... 9 more Caused by: java.lang.IllegalArgumentException: Output path does not exist or is not writeable: gs://tempxyz at org.apache.beam.sdk.repackaged.com.google.common.base.Preconditions.checkArgument(Preconditions.java:191) at org.apache.beam.sdk.util.GcsPathValidator.verifyPathIsAccessible(GcsPathValidator.java:78) at org.apache.beam.sdk.util.GcsPathValidator .validateOutputFilePrefixSupported(GcsPathValidator.java:62) at org.apache.beam.sdk.options.GcpOptions$GcpTempLocationFactory.create(GcpOptions.java:217) ... 15 more

Answer 1

Somewhere in the documentation it was mentioned that I need to run the following .

gcloud auth application-default login

Obviously the exception does not help with figuring out this , I hope it helps someone else.