在 Dataproc 的集群中查找 Hadoop 流式传输 jar

Question

So I want to run a Python map reduce job on a Dataproc cluster, the problem is that I cannot find the Hadoop streaming jar file that needs to be submitted into the Main class or jar input.

我正在使用带有 Hadoop 2.7 的 Dataproc 映像，Hadoop 2.8 的答案也可以。

谢谢

Answer 1

Hadoop 流 jar 位于 Dataproc 集群节点上的/usr/lib/hadoop-mapreduce/文件夹中： /usr/lib/hadoop-mapreduce/hadoop-streaming.jar