Rabbitmq异步生产者脚本

Question

RabbitMQ newbie here, I hope you can help me solve a problem:

I have a third-party program which I want use as a "producer" for RabbitMQ. It generates a lot of log messages, and can only send them out by triggering a shell script.

I can write a shell script using python and pika, but AFAI understand, that will create a new connection for every single message sent to the RabbitMQ server. That seems like a bad idea--the reason I'm switching to RabbitMQ in the first place was because my previous solution (wget posts) wasn't fast enough to keep up with the outflow of logs.

I'm sure there's a right way to do this, but I can't figure it out. I have to call the script every time I have a log to send--I don't know how to persist the connection, or if I'm even thinking in the right direction.

Thanks in advance.

Answer 1

You can use same connection for multiple operations. Normally we reuse it.

But if script called to send every message than yupp, it will create (and close) new connection every time, which is huge overhead. So some daemon script which establish connection and wait on local port for messages to proxy to rabbitmq may come in handy.

Alternatively, maybe some kind of logstash-forwarder can solve your problem?

Also, give RabbitMQ Python examples closer look if you new to it and have not seeing it yet.