Can I start and stop delayed_job workers from within my Rails app?

Question

I've got an app that could benefit from delayed_job and some background processing. The thing is, I don't really need/want delayed_job workers running all the time.

The app runs in a shared hosting environment and in multiple locations (for different users). Plus, the app doesn't get a large amount of usage.

Is there a way to start and stop processing jobs (either with the script or rake task) from my app only after certain actions/events?

Answer 1

You could call out to system:

system "cd #{Rails.root} && rake delayed_job:start RAILS_ENV=production"

You could just change delayed_job to check less often too. Instead of the 5 second default, set it to 15 minutes or something.

Answer 2

If you go to stop a worker, you are given the PID. You can simply kill -9 PID if all else fails.

Answer 3

Yes, you can, but I'm not sure what the benefit will be. You say you don't want workers running all the time - what are your concerns? Memory usage? Database connections?

To keep the impact of delayed_job low on your system, I'd run only one worker, and configure it to sleep most of the time.

Delayed::Worker::sleep_delay = 60 * 5  # in your initializer.rb

A single worker will only wake up and check the db for new jobs every 5 minutes. Running this way keeps you from 'customizing' too much.

But if you really want to start a Delayed::Worker programatically, look in that class for work_off , and implement your own script/run_jobs_and_exit script. It should probably look much like script/delayed_job does - 3 lines.

Answer 4

I found this because I was looking for a way to run some background jobs without spending all the money to run them all the time when they weren't needed. Someone made a hack using google app engine to run the background jobs:

http://viatropos.com/blog/how-to-run-background-jobs-on-heroku-for-free/

It's a little outdated though. There is an interesting comment in the thread:

"When I need to send an e-mail, copy a file, etc I basically add it to the queue. At the end of every request it checks if there is anything in the queue. If so then it uses the Heroku API to set the worker to 1. At the end of a worker getting a task done it checks to see if there is anything left in the queue. If not then it sets the workers back to 0. The end result is the background worker will just work for a few seconds here and there. I can do all the background processing that I need and the bill at the end of the month rarely ever reaches 1 hour total worth of work. Even if it does no problem, I'll pay $0.05 for background processing. :)"