简体繁体 English

是PYTHON Gearman Worker接受多任务

[英]is PYTHON Gearman Worker accept multi-tasks

原文 2012-04-13 02:45:15 8 1 gearman/ python-gearman

For example: I have a task named "URLDownload", the task's function is download a large file from internet. 例如：我有一个名为“URLDownload”的任务，任务的功能是从互联网上下载一个大文件。 Now I have a Worker Process running, but have about 1000 files to download. 现在我正在运行一个Worker Process，但是要下载大约1000个文件。 It is easy for a Client Process to create 1000 task, and send them to Gearman Server. 客户端进程很容易创建1000个任务，并将它们发送到Gearman Server。

My Question is the Worker Process will do the task one by one, or it will accept multi-tasks at one time, 我的问题是工人流程将逐个完成任务，或者它将同时接受多任务，

If the Worker Process can accept multi-tasks, So How can I limit the task-pool-size in Worker Process. 如果工作进程可以接受多任务，那么我如何限制工作进程中的任务池大小。

1 个解决方案

Workers process one request at a time. 工人一次处理一个请求。 You have a few options: 你有几个选择：

1) You can run multiple workers (this is the most common method). 1）您可以运行多个工作程序（这是最常用的方法）。 Workers sit in poll() when they aren't processing so this model works pretty well. 当工人没有处理时，工人坐在poll（）中，所以这个模型运行得很好。

2) Write a fork() implementation around the worker. 2）在worker周围编写一个fork（）实现。 This way you can fire up a set number of worker processes, but don't have to monitor multiple processes. 这样，您可以启动一定数量的工作进程，但不必监视多个进程。