How do I use “if __name__ == '__main__':” in a .bat file?

Question

I have a multiprocessing function that runs fine in Spyder like this:

if __name__ == '__main__':
    global results
    p = Pool(20)
    results = p.map(get_api_item, date_list)
    p.terminate()
    p.join()

result = pd.concat(results)

path = r'<path>'
result.to_csv(os.path.join(path,r'api_item.csv'), index=False)

When I run that .py file in a .bat file...

@echo off
"E:\Python\Anaconda3\python.exe" "E:\Projects\api.py" %*
pause

...I get this error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_ma
in
    exitcode = _main(fd)
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "E:\Python\Anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_m
ain_from_path
    run_name="__mp_main__")
  File "E:\Python\Anaconda3\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "E:\Python\Anaconda3\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "E:\Python\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "<path to api>\api.py", line 29, in <module>
    result = pd.concat(results)
NameError: name 'results' is not defined

I think this is because the service (.bat file) is likely not calling it __main__ . How do I remedy this?

Answer 1

Please read and understand the warning in the mutiprocessing docs ( https://docs.python.org/3.8/library/multiprocessing.html ) starting at:

Safe importing of main module

Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).

Your module will be imported by other processes, so you cannot have code at the global (outermost) level depending on code in your __main__ block.