Hadoop MapReduce how to pass a string argument to the Mapper

Question

I have been using a Hadoop MapReduce command on SSH like this:

hadoop jar /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.0.1.jar -file /nfs_home/appers/user1/mapper.py -file /nfs_home/appers/user1/reducer.py -mapper '/usr/lib/python_2.7.3/bin/python mapper.py' -reducer '/usr/lib/python_2.7.3/bin/python reducer.py' -input /ccexp/data/test_xml/0901282-510179094535002-oozie-oozi-W/extractOut//.xml -output /user/ccexptest/output/user1/MRoutput

However, I am expanding the capability of the Mapper.py script and would like to pass a string argument to the Mapper when launching the MapReduce job. How can I edit the above MR command so that a string is included as an argument to the Mapper?

Answer 1

您可以根据情况将任意命令行输入传递给-mapper和-reducer 。

-mapper '/usr/lib/python_2.7.3/bin/python mapper.py myargs'