流星的背景任务

Question

I'm wondering, is there is way to implement background taks, maybe with workers pool. Can you show me direction, i'm thinking about writing package for this?

Answer 1

2019 update

Before thinking about writing a package for anything, first look if there are existing packages that do what you need. In the Meteor world, this means looking on Atmosphere for "job/queue/task/worker management/scheduling" packages, then on npm for the same search terms. You also need to define your requirements more precisely:

• do you want persistence, or would an in-memory solution work?
• do you want to be able to distribute jobs to different machines?

Meteor-specific

• job-collection - reliable (I used it in 2014 in production at a startup), but currently in maintenance mode. Lets you schedule persistent jobs to be run anywhere (servers, clients).
• SteveJobs - actively maintained by Max Savin, the author of several powerful Meteor tools
• littledata:synced-cron - "A simple cron system for Meteor. It supports syncronizing jobs between multiple processes."

Abandoned packages:

• artwells:queue - priorities, scheduling, logging, re-queuing. Queue backed by MongoDB. Last code commit : 2015-Oct.
• super basic cron packages: easycron . Last update: Dec 2015.
• differential:workers - Spawn headless worker meteor processes to work on async jobs. Last code commit : Jan 2015
• cron ( since 2015 )
• PowerQueue - abandoned since 2014 . Queue async tasks, throttle resource usage, retry failed. Supports sub queues. No scheduling . No tests, but nifty demo . Not suitable for running for a long while due to using recursive calls .

Npm packages

Meteor has been able to use npm packages directly for several years now, so this question amounts to finding job/worker/queue management packages on NPM. If you don't care about persistence:

• Async "provides around 70 functions that include the usual 'functional' suspects ( map , reduce , filter , each ...) as well as some common patterns for asynchronous control flow ( parallel , series , waterfall ...)"
• d3-queue - minimalistic, written by D3 author Mike Bostock

If you do want persistence, since Meteor uses MongoDB already, it may be advantageous to use a job scheduling package with persistence to MongoDb. The most powerful and popular seems to be Agenda , but unfortunately it hasn't been maintained in months, and it has a significant backlog of issues .

If you're willing to add a dependency backed by redis to your project, there are more choices:

• bull - the most full-featured job queue solution for Node, backed by Redis
• bee - simple, fast, robust. Does not suffer from a memory leak that Bull exhibits
• Kue - priority job queue for Node

Like MongoDB, Redis can also provide high-availability (via Redis Sentinel), and if you want to distribute jobs among multiple worker machines, you can point them all at the same Redis server .

Answer 2

There is a package based on Cron jobs which can be used to schedule tasks on certain intervals or dates. Here is the package: https://atmosphere.meteor.com/package/cron

And if you happen to look into the source of that package, you'll notice they're simply using:

Meteor.setInterval( ... , delay );

So if you save your tasks in a database, then load them into intervals during startup, then you'll probably be on the right track.

Answer 3

If you are looking for something that is specific to Meteor, I am happy to share there is a new package called Steve Jobs. It makes running background jobs as easy as calling a Method.

It has all the standard features you would expect, such as running a job only once, retrying failed jobs, and so on. You can learn more about it on GitHub:

http://github.com/msavin/stevejobs

Answer 4

I'm guessing proper support is on their roadmap, but in the meantime I've managed to get it going in a hacky way via setInterval . See the cron-tick package.