简体繁体 English

按消费者分配芹菜工人

[英]distributing celery workers by consumer

原文 2019-01-22 15:44:23 8 1 python/ rabbitmq/ celery

here is a little background for my question.这是我的问题的一些背景。
We have a converter that converts xml to api calls.我们有一个转换器，可以将 xml 转换为 api 调用。
we have a lot of customers converting xml's and now some customers have to wait until a customer finished converting 100's of xml files.我们有很多客户在转换 xml，现在有些客户必须等到客户完成 100 个 xml 文件的转换。
we have celery working with rabbitmq我们有 celery 和 rabbitmq 一起工作

what i'm trying to achieve is:我想要实现的是：
assign all celery workers to one customer until another customer sends xml's if there is another customer in the queu assign half the celery workers to another customer.将所有 celery 工作人员分配给一个客户，直到另一个客户发送 xml，如果队列中有另一位客户，则将一半的 celery 工作人员分配给另一位客户。

Can someone point me in the right direction, i can't seem to figure out where to start looking.有人能指出我正确的方向吗，我似乎不知道从哪里开始寻找。 Sorry for my weird sentences english is not my first language.对不起，我的奇怪句子英语不是我的第一语言。

1 个解决方案

About what your are trying to achieve - too complicated for Celery, I think.关于您想要实现的目标 - 我认为对于 Celery 来说太复杂了。

The obvious answer is to increase number of Celery workers using concurrency option while starting worker.显而易见的答案是在启动 worker 时使用concurrency选项增加 Celery worker 的数量。 It is easy to do and you do not need to change your code.这很容易做到，您无需更改代码。 I would recommend it.我会推荐它。

Another option that is not so easy, however, can be a beautiful solution, is to use separate RabbiMQ virtual host for each customer.然而，另一个不太容易的选项可能是一个漂亮的解决方案，是为每个客户使用单独的RabbiMQ 虚拟主机。 I can't tell you how to do it completely in this case, but it could your starting point.在这种情况下，我无法告诉您如何完全做到这一点，但它可以是您的起点。