蝗虫将响应保存在文件中

Question

I'm using the event hook "request_success" to store each response in a file.

class Print:  # pylint: disable=R0902
"""
Record every response (useful when debugging a single locust)
"""

def __init__(self, env: locust.env.Environment, include_length=False, include_time=False):
    self.env = env
    self.env.events.request_success.add_listener(self.request_success)

def request_success(self, request_type, name, response_time, response_length, **_kwargs):
    users = self.env.runner.user_count
    data = [datetime.now(), request_type, name, response_time, users]
    state_data.append(data)


@events.init.add_listener
def locust_init_listener(environment, **kwargs):
    Print(env=environment)


@events.quitting.add_listener
def write_statistics(environment, **kwargs):
    with open("output/requests_stats_u150_c.csv", "a+") as f:
        csv_writer = csv.writer(f)
            for row in state_data:
            csv_writer.writerow(row)

However, for multiple workers, some of the lines overlap, I even lost some of the lines. Here's the sample error response.

2021-03-18 17:08:28.019587,POST,login,262.42033099697437,50
2021-03-18 17:08:28.021776,POST,select_order,16.014199994970113,50
2021-03-18 17:08:28.028505,GET2021-03-18 17:08:28.030823,GET,home,3.924126998754218,50

The 3rd entry got corrupted by double entry. I guess both of the worker tried to write at once.

Any idea how can I store all successful responses in a multi-worker locust test?

Answer 1

It really depends on exactly what data you need. The workers already send all the request data to the master and the master has built-in ability to save stats to CSV . If you need more than this, I would still have the master do it. Much easier to control writing to a file from one place than coordinate multiple.

You can use report_to_master event on the workers to add in whatever extra data you may need to be included in the payload reported to the master. Then on the master you can use worker_report for the master to pull out the data from the worker payloads. I'd probably save it to some other variables on the master and then have another function periodically write the data to a file so there's no contention in writes. You could start the writing by using the init hook, spawn your own greenlet that writes the data and then sleeps for 2 seconds (that's the interval that workers report to master at).