Locust 脚本被 ThreadedConnectionPool 卡住了

Question

I'm trying to write a load testing script using locust in python (it uses gevent & greenlet for multithreading internally as per my understanding); but my script is getting stuck when I try to put a db connection (postgres) back to connection pool inside a thread. I have defined connection pool variable as gloabl & then trying to create & put connections back inside threads; I have no experience in threading; not sure if these lines marked inside ** quotes are the reason locust gets stuck -

@events.test_start.add_listener # executes one time at the start of test run
def on_test_start(**kw):
    **global t_pool**  # connection pool variable defined as global
    conn_st = config(fname = os.path.join('.', 'db'), instance = 'xxx')
    try:
        t_pool = pool.ThreadedConnectionPool(1, 100, **conn_st)
        #cur = db.cursor()
    except Exception as err:
        #db = None
        raise DatabaseError("DB Connection could not be established - %s" %(err))

@contextmanager
def get_con():
    con = t_pool.getconn()
    try:
        yield con
    finally:
        **t_pool.putconn(con)**

def send_xxx():
    ind = random.randint(0, (len(xxx_data['yyy'])-1))
    body['toeknx'] = xxx_data['yyy'][ind]
    res = requests.post(url, headers=headers, json=body)
    res.raise_for_status()
    with get_con() as con:
        cur = con.cursor()
        while True:
            cur.execute("test query")
            token = cur.fetchone()
            if token is None:
                continue
            else:
                cur.execute("another test query")
                out = cur.fetchone()
                if zzz:
                    continue
                else:
                    cur.execute("final test query")
                    out1 = cur.fetchone()
                    time_diff = out1[1] - out1[0]
                    cur.close()

Answer 1

You need to use psycogreen to make psycopg2 gevent-friendly.

Something like:

import gevent
import gevent.monkey

gevent.monkey.patch_all()
import psycogreen.gevent

psycogreen.gevent.patch_psycopg()

Full example here: https://github.com/SvenskaSpel/locust-plugins/blob/master/locust_plugins/listeners.py