在 Dataproc 的集群中查找 Hadoop 流式传输 jar

Question

So I want to run a Python map reduce job on a Dataproc cluster, the problem is that I cannot find the Hadoop streaming jar file that needs to be submitted into the Main class or jar input.

I'm using Dataproc image with Hadoop 2.7, an answer for Hadoop 2.8 would also work.

Thanks

Answer 1

Hadoop Streaming jar located in /usr/lib/hadoop-mapreduce/ folder on Dataproc cluster nodes: /usr/lib/hadoop-mapreduce/hadoop-streaming.jar