Python - 如何将单独的模块（不运行）作为单独的进程运行？

Question

tl,dr: How can I programmably execute a python module (not function) as a separate process from a different python module?

On my development laptop, I have a 'server' module containing a bottle server. In this module, the name==main clause starts the bottle server.

@bt_app.post("/")
def server_post():
    << Generate response to 'http://server.com/' >>

if __name__ == '__main__':
    serve(bt_app, port=localhost:8080)

I also have a 'test_server' module containing pytests. In this module, the name==main clause runs pytest and displays the results.

def test_something():
        _rtn = some_server_function()
        assert _rtn == desired

if __name__ == '__main__':
    _rtn = pytest.main([__file__])
    print("Pytest returned: ", _rtn)

Currently, I manually run the server module (starting the web server on localhost), then I manually start the pytest module which issues html requests to the running server module and checks the responses.

Sometimes I forget to start the server module. No big deal but annoying. So I'd like to know if I can programmatically start the server module as a separate process from the pytest module (just as I'm doing manually now) so I don't forget to start it manually.

Thanks

Answer 1

There is my test cases dir tree:

test
├── server.py
└── test_server.py

server.py start a web server with flask.

from flask import Flask                                                                                                                                                
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'


if __name__ == '__main__':
    app.run()

test_server.py make request to test.

import sys                                                                                                                                                             
import requests
import subprocess
import time

p = None  # server process

def start_server():
    global p
    sys.path.append('/tmp/test')
    # here you may want to do some check. 
    # whether the server is already started, then pass this fucntion
    kwargs = {}  # here u can pass other args needed
    p = subprocess.Popen(['python','server.py'], **kwargs)

def test_function():
    response = requests.get('http://localhost:5000/')
    print('This is response body: ', response.text)

if __name__ == '__main__':
    start_server()
    time.sleep(3)  # waiting server started
    test_function()
    p.kill()

Then you can do python test_server to start the server and do test cases.

PS: Popen() needs python3.5+. if older version, use run instead

Answer 2

import logging
import threading
import time

def thread_function(name):
    logging.info("Thread %s: starting", name)
    time.sleep(2)
    logging.info("Thread %s: finishing", name)

if __name__ == "__main__":
    format = "%(asctime)s: %(message)s"
    logging.basicConfig(format=format, level=logging.INFO,
                        datefmt="%H:%M:%S")

    threads = list()
    for index in range(3):
        logging.info("Main    : create and start thread %d.", index)
        x = threading.Thread(target=thread_function, args=(index,))
        threads.append(x)
        x.start()

    for index, thread in enumerate(threads):
        logging.info("Main    : before joining thread %d.", index)
        thread.join()
        logging.info("Main    : thread %d done", index)

With threading you can run multiple processes at once!

Answer 3

Wim baasically answered this question. I looked into the subprocess module. While reading up on it, I stumbled on the os.system function.

In short, subprocess is a highly flexible and functional program for running a program. os.system, on the other hand, is much simpler, with far fewer functions.

Just running a python module is simple, so I settled on os.system.

import os
server_path = "python -m ../src/server.py"
os.system(server_path)

Wim, thanks for the pointer. Had it been a full fledged answer I would have upvoted it. Redo it as a full fledged answer and I'll do so.

Answer 4

Async to the rescue.

import gevent
from gevent import monkey, spawn
monkey.patch_all()
from gevent.pywsgi import WSGIServer

@bt_app.post("/")
def server_post():
    << Generate response to 'http://server.com/' >>

def test_something():
    _rtn = some_server_function()
    assert _rtn == desired
    print("Pytest returned: ",_rtn)
    sleep(0)

if __name__ == '__main__':
    spawn(test_something) #runs async
    server = WSGIServer(("0.0.0.0", 8080, bt_app)
    server.serve_forever()