I'm currently in the market for a new queue system for jobs we have in our system. I've tried beanstalk but it's been unable to keep up with the load. I'm looking for a simple system to get up and running that I can put pieces of data in from producers and have consumers in Java, PHP and Python pull data off and process it.
Ideally I'd like to see features such as:
Job verification -> jobs are removed from the queue only when I've marked them as finished (in case of failures I don't have to put the jobs back in)
Priorities -> ability to prioritize jobs
Multiple channels -> ability to have one queue that can service several apps with separate data streams(or databases)
Disk Persistence -> ability to have jobs written to disk in case of failures
anyone have any good suggestions?
Currently looking at RabbitMQ
How about Apache ActiveMQ .
Accessible from Java, PHP, Python.
Supports all the features you requested.
RabbitMQ是一个很好的消息传递系统,它有Java,PHP,Python和许多其他语言的绑定。
The Berkeley database can be used to build a priority queue with bindings to most relevant languages. The HA (High Availability) configuration can make it distributed as well. I believe the Sun Grid Engine, for instance, uses just that to synchronize jobs.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.