Python外部项目脚本执行

Question

After many researches on the web, on many different topics, I wasn't able to find the solution answering my problem.

I am developing a tool (with graphic interface) allowing to process python scripts from different projects. To not have any problem of dependences, I ask the user to target the project directory as well as the script to process.

Thus to develop this tool I have in my IDE a project "benchmark_tool" which is the project integrating the GUI, and the project "test_project" which groups various testing scripts. Obviously, the final goal is the tool to be compiled and thus that the project "benchmark_tool" disappears from IDE.

Here is the IDE architecture:

benchmark_tool (python project)
    __init__.py
    main.py

test_project (python project)
    __init__.py
    module1
        __init__.py
        test_script.py
    module2
        __init__.py
        imports.py

How it works: The main.py shall call test_script.py. test_script.py calls imports.py at the first beggining of the script.

UPDATE

I tried some modifications in my main code:

import sys
import subprocess

subprocess.check_call([sys.executable, '-m', 'test_project.module1.test_script'], cwd='D:/project/python')

I got this error

Traceback(most recent call last):
    File "C:/Python31/lib/runpy.py", line 110, in run module as main
        mod_name, loader, code, fname = _get_module_detail(mod_name)
    File "C:/Python31/lib/runpy.py", line 91, in get module details
        code = loader.get_code(mod_name)
    File "C:/Python31/lib/pkgutil.py", line 272, in get code
        self.code = compile(source, self.filename, 'exec')
    File "D:/project/python/test_project/module1/test_script.py", line 474


SyntaxError: invalid syntax
Traceback (most recent call last):
    File "D:/other_project/benchmark_tool/main.py", line 187, in read
        subprocess.check_call([sys.executable, '-m', 'module1.test_script.py'], cwd='D:/project/python/test_project')
    File "C:/Python31/lib/subprocess.py", line 446, in check call
        raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:/Python31/python.exe', '-m', 'module1.test_script.py']' returned non-zero exit status 1

Note It works with subprocess.Popen(['python','D:/project/python/test_project/module1/test_script.py])

What's the main difference between both methods ? I'll also have to pass arguments to test_scripts.py, which one is the best to use to communicate with a python script (input and output datas are exchanged) ?

Thanks by advance

Answer 1

To run an "external project script", install it eg: pip install project (run it in a virtualenv if you'd like) and then run the script as you would with any other executable:

#!/usr/bin/env python
import subprocess

subprocess.check_call(['executable', 'arg 1', '2'])

If you don't want to install the project and it has no dependencies (or you assume they are already installed) then to run a script from a given path as a subprocess (assuming there is module1/__init__.py file):

#!/usr/bin/env python
import subprocess
import sys

subprocess.check_call([sys.executable, '-m', 'module1.test_script'], 
                      cwd='/abs/path/to/test_project')

Using -m to run the script, avoids many import issues .

The above assumes that test_project is not a Python package (no test_project/__init__.py ). If test_project is itself a Python package then include it in the module name and start from the parent directory instead:

subprocess.check_call([sys.executable, '-m', 'test_project.module1.test_script'], 
                      cwd='/abs/path/to')

You could pass the path to test_project directory as a command-line parameter ( sys.argv ) or read it from a config file ( configparser , json ). Avoid calculating it relative to the installation path of your parent script (if you have to then use pkgutil , setuptools' pkg_resources , to get the data). To find a place to store user data, you could use appdirs Python package .

Answer 2

There are 3 problems I see that you have to fix:

1. In order to import from module2, you need to turn that directory into a package by placing an empty *__init__.py file in it
2. When executing test_script.py , you need the full path to the file, not just file name
3. Fixing up sys.path only work for your script. If you want to propagate that to test_script.py , set the PYTHONPATH environment variable

Therefore, my solution is:

import os
import sys
import subprocess

# Here is one way to determine test_project
# You might want to do it differently
script_dir = os.path.abspath(os.path.dirname(__file__))
test_project = os.path.normpath(os.path.join(script_dir, '..', 'test_project'))

python_path = '{}:{}'.format(os.environ.get('PYTHONPATH', ''), test_project)
python_path = python_path.lstrip(':')  # In case PYTHONPATH was not set

# Need to determine the full path to the script
script = os.path.join(test_project, 'module1', 'test_script.py')
proc = subprocess.Popen(['python', script], env=dict(PYTHONPATH=python_path))