简体繁体 English

PHP多线程RabbitMQ使用者

[英]Multi-threading RabbitMQ consumers with PHP

原文 2014-08-03 05:28:48 0 1 php/ multithreading/ multiprocessing/ amqp/ supervisord

I'm developing an app which uses rabbitmq for queuing tasks. 我正在开发一个使用Rabbitmq进行任务排队的应用程序。 I have a worker script which listen to the queue and do the job. 我有一个工作脚本，可以监听队列并完成工作。 What I'm wondering is how to manage multiple consumers in parallel. 我想知道的是如何并行管理多个消费者。 I initially thought of using pthreads for multi-threading, but ran into all kind of weird issues, and for what I've read in its Github page, it's not possible, something about AMQP extension not being thread safe. 我最初考虑使用pthreads进行多线程处理，但是遇到了各种奇怪的问题，对于我在Github页上阅读的内容，这是不可能的，关于AMQP扩展的问题不是线程安全的。

I also read something about Supervisord (haven't really try it though) but what I read suggested me I can't just change dynamically modify the numprocs without restarting the other processes. 我还读了一些有关Supervisord的文章（虽然还没有真正尝试过），但是我读到的书暗示我不能在不重新启动其他进程的情况下进行动态更改以修改numproc。 This would actually be on demand, having some backend and when some variables are modified, the number of workers is modified. 这实际上是按需的，具有一些后端，并且在修改某些变量时，会修改工人的数量。

Now, my best shot is to run multiple processes of the worker script. 现在，我最好的选择是运行工作脚本的多个进程。 But how to manage them? 但是如何管理它们呢？ One of the main issues is that the number of workers needed may vary over time, and how to ensure that if the server crashes, they will be up and running asap. 主要问题之一是所需的工作人员数量可能会随时间变化，以及如何确保如果服务器崩溃，他们将尽快启动并运行。

Is there some sort of recommended pattern for doing this? 有某种推荐的模式可以做到这一点吗？ I thought of running a script with upstart or maybe a cron, which forks the processes, but then when it run again it would have to check if the number of worker's available suffice, and to be able to do that I think I would have to store the PIDs in a database. 我考虑过用新贵或cron运行一个脚本，该脚本会分叉进程，但随后再次运行时，它必须检查该工人的可用数量是否足够，并且能够做到这一点，我认为我必须这样做将PID存储在数据库中。 I guess there may be another way. 我想可能还有另一种方式。

UPDATE: I thought of a way to use Supervisord and modifying the numprocs in order to run new workers when needed (this would be actually rare but need to cover the scenario). 更新：我想到了一种使用Supervisord并修改numproc的方法，以便在需要时运行新的工作程序（这实际上很少见，但需要涵盖该场景）。 It would work this way: Whenever I need to run new workers, I would send a message to the current workers to stop listening to the work queue, when they are done doing their current tasks, I restart the workers, including the new ones. 它将以这种方式工作：每当我需要运行新工作人员时，我都会向当前工作人员发送一条消息以停止监听工作队列，当他们完成当前任务后，我将重新启动工作人员，包括新工作人员。

What do you think about it? 你怎么看待这件事？ I know it's probably not the best design, but it's the best I can do. 我知道这可能不是最好的设计，但这是我能做的最好的。 I have some doubts on how to implement the part of sending the message with rabbitmq (first time working with a message broker) but I think I can handle it 我对如何实现通过rabbitmq发送消息（第一次使用消息代理）有一些疑问，但我认为我可以处理

1 个解决方案

Yes, you can use multiple processes (php scripts) controlled by supervisord (or monit ) or you may utilize threading, but in this case you have to open new connection (not only channel) per each thread to prevent various errors (with both php-amqp extension and php-amqplib library). 是的，您可以使用由supervisored （或monit ）控制的多个进程（php脚本），也可以使用线程，但是在这种情况下，您必须为每个线程打开新的连接（不仅是通道），以防止发生各种错误（两个php都可以） -amqp扩展和php-amqplib库）。

Keep in mind that PHP was not designed to manage resources in threaded environment, so multiprocess way looks more stable and maintainable than multithreaded. 请记住，PHP并非旨在在线程环境中管理资源，因此多进程方式看起来比多线程更稳定和可维护。