在 Dataproc 的集群中查找 Hadoop 流式傳輸 jar

Question

So I want to run a Python map reduce job on a Dataproc cluster, the problem is that I cannot find the Hadoop streaming jar file that needs to be submitted into the Main class or jar input.

我正在使用帶有 Hadoop 2.7 的 Dataproc 映像，Hadoop 2.8 的答案也可以。

謝謝

Answer 1

Hadoop 流 jar 位於 Dataproc 集群節點上的/usr/lib/hadoop-mapreduce/文件夾中： /usr/lib/hadoop-mapreduce/hadoop-streaming.jar