无法从 Locust 执行 RMQ 发布“BlockingIOError：[WinError 10035]”

Question

My project requires client sending messages directly to Rabbit MQ and we need to do load testing for this.

I tried PIKA, works fine in a plain python file but as soon as I tried to implement this in Locust I start getting error due to compatibility issues

I tried PIKA Async, B-Rabbit, etc.. none works with Locust(Gevent)

I dont have to integrate with locust but just importing locust on these python file is enough to trigger the error.

I have read in several blogs that Gevent is not compatible with pika.

class RMQ:

    def __init__(self) -> None:
        self.connection = pika.BlockingConnection(pika.ConnectionParameters('localhost', credentials=pcredentails))
        self.channel = self.connection.channel()

    def connect(self):
        self.channel.basic_publish(exchange='locust_events', routing_key='python3', body='Hello World!')
        print("[x] Sent 'Hello World!'")

    def close(self):
        self.channel.close()
        self.connection.close()

Error:

BlockingIOError: [WinError 10035] A non-blocking socket operation could not be completed immediately

Some one please let me know a possible way to solve this

Note: B-rabbit did say it is thread safe but it still throws error when I publish "Time out reading from server" with 12s delay, this happens only when I use locust else it is fast

Answer 1

Pika has a GeventConnection connection class. That is what you should be using.

---

_{the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.}

Answer 2

Having tried

**Pika** = not compatible with Gevent Locust (esp windows)

**B-rabbit**, **Rabbit-py** = Slow with locust and times out

I can confirm Kombu works perfectly with Locust, anyone looking to implement queues with locust this is the solution

https://github.com/celery/kombu

Answer 3

@luke Bakken:

Im using my own bespoke framework for my company, its not so simple to decouple this bit.. so I created a quick snippet class for Kombu

from locust import HttpClient, TaskSet, task, SingleRunner from kombu import Connection, Exchange, Queue

class KombuClient(HttpClient):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.connection = Connection('amqp://guest:guest@localhost:5672//')
        self.channel = self.connection.channel()

    def send_message(self, message):
        exchange = Exchange('test_exchange', type='direct')
        queue = Queue('test_queue', exchange, routing_key='test_key')
        queue.maybe_bind(self.connection)
        queue.declare()

        producer = self.connection.Producer(exchange=exchange, routing_key='test_key')
        producer.publish(message)
        print(" [x] Sent message: '{}'".format(message))

    def close_connection(self):
        self.connection.release()

class UserBehavior(TaskSet):
    @task
    def send_hello_message(self):
        self.client.send_message('Hello World!')

class WebsiteUser(HttpUser):
    tasks = [UserBehavior]
    min_wait = 5000
    max_wait = 9000
    client = KombuClient

def main():
    runner = SingleRunner(http_user_cls=WebsiteUser)
    runner.run(options={
        "host": "http://example.com",
        "num_users": 1000,
        "hatch_rate": 100
    })

if __name__ == "__main__":
    main()