无法在本地运行python MRJob

Question

If i understand MRJob correctly, you can simulate hadoop's multi process run using MRJob by running it with

python mrfile.py -r local input.txt

I'm running windows(no choice for now), and when I issue the above command, i'm getting a bunch of mambo jumbo and at the end it tells me :

WindowsError: [Error 2] The system cannot find the file specified

This is the full error. Could someone help?

C:\Users\someuser\Documents\Python_projects\something>python MRJob_parser.py -r loc
al test2.txt
no configs found; falling back on auto-configuration
no configs found; falling back on auto-configuration
creating tmp directory c:\users\someuser\appdata\local\temp\MRJob_parser.someuser.
20150701.211822.496000
writing wrapper script to c:\users\someuser\appdata\local\temp\MRJob_parser.bw401
45.20150701.211822.496000\setup-wrapper.sh
writing to c:\users\someuser\appdata\local\temp\MRJob_parser.someuser.20150701.211
822.496000\step-0-mapper_part-00000
> sh -ex setup-wrapper.sh 'c:\Users\someuser\Documents\python_venv\something_Project\
Scripts\python.exe' MRJob_parser.py --step-num=0 --mapper 'c:\users\someuser\appd
ata\local\temp\MRJob_parser.someuser.20150701.211822.496000\input_part-00000' > c
:\users\someuser\appdata\local\temp\MRJob_parser.someuser.20150701.211822.496000\s
tep-0-mapper_part-00000
Traceback (most recent call last):
  File "MRJob_parser.py", line 18, in <module>
    Extractor.run()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\job.py", line 461, in run
    mr_job.execute()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\job.py", line 479, in execute
    super(MRJob, self).execute()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\launch.py", line 151, in execute
    self.run_job()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\launch.py", line 214, in run_job
    runner.run()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\runner.py", line 464, in run
    self._run()
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\sim.py", line 173, in _run
    self._invoke_step(step_num, 'mapper')
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\sim.py", line 260, in _invoke_step
    working_dir, env)
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\local.py", line 147, in _run_step
    procs_args, output_path, working_dir, env)
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\local.py", line 250, in _invoke_processes
    cwd=working_dir, env=env)
  File "c:\Users\someuser\Documents\python_venv\something_Project\lib\site-packages\m
rjob\local.py", line 73, in _chain_procs
    proc = Popen(args, **proc_kwargs)
  File "C:\Python27\Lib\subprocess.py", line 710, in __init__
    errread, errwrite)
  File "C:\Python27\Lib\subprocess.py", line 958, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

C:\Users\someuser\Documents\Python_projects\something>

Answer 1

From https://github.com/Yelp/mrjob :

# locally
python mrjob/examples/mr_word_freq_count.py README.rst > counts

My guess is that you don't have to pass in the local runner option and that's causing some kind of error.