Rails: managing multiple sidekiqs without upstart script

Question

Once upon a time, I had one app - Cashyy . It used sidekiq. I deployed it and used this upstart script to manage sidekiq (start/restart).

I decide to deploy another app to the same server. The app (let's call it Giimee ) also uses sidekiq.

And here is the issue. Sometimes I need to restart sidekiq for Cashyy , but not for Giimee . Now, as I understand I will need to hack something using index thing (in upstart script and managing sidekiqs: sudo restart sidekiq index=1 ) (if I understood it correctly).

BUT!

I have zero desire to dabble with these indexes (nightmare to support? Like you need to know how many apps are using sidekiq and be sure to assign unique index to each sidekiq. And to know assigned index if you want to restart specific sidekiq).

So here is the question: how can I isolate each sidekiq (so I would not need to maintain the index ) and still get the stability and usability of upstart (starting the process, restarting, etc.)?

Or maybe I don't understand something and the thing with index is state of art?

Answer 1

As an alternative to an upstart script, you can use Capistrano and Capistrano-Sidekiq to manage those Sidekiqs.

We have Sidekiq running on 3 machines and have had a good experience with these two libraries/tools.

Note : we currently use an older version of Capistrano (2.15.5)

In our architecture, the three machines are customized slightly on deploy. This led us to break up our capistrano deploy scripts by machine so that we could customize some classes, manage Sidekiq, etc. Our capistrano files are structured something like this:

- config/
  - deploy.rb
  - deploy/ 
    - gandalf.rb
    - gollum.rb
    - legolas.rb

With capistrano-sidekiq, we are able to control, well, Sidekiq :) at any time (during a deploy or otherwise). We set up the Sidekiq aspects of our deploy scripts in the following way:

# config/deploy.rb
# global sidekiq settings
set :sidekiq_default_hooks, false
set :sidekiq_cmd, "#{fetch(:bundle_cmd, 'bundle')} exec sidekiq"
set :sidekiqctl_cmd, "#{fetch(:bundle_cmd, 'bundle')} exec sidekiqctl"    
set :sidekiq_role, :app
set :sidekiq_pid, "#{current_path}/tmp/pids/sidekiq.pid"
set :sidekiq_env,  fetch(:rack_env, fetch(:rails_env, fetch(:default_stage)))
set :sidekiq_log,  File.join(shared_path, 'log', 'sidekiq.log')

# config/deploy/gandalf.rb 
# Custom Sidekiq settings 
set :sidekiq_timeout, 30
set :sidekiq_processes, 1
namespace :sidekiq do
  # .. code omitted from methods and tasks for brevity
  def for_each_process(&block)   
  end

  desc 'Quiet sidekiq (stop accepting new work)'
  task :quiet, :roles => lambda { fetch(:sidekiq_role) }, :on_no_matching_servers => :continue do
  end

  desc 'Stop sidekiq'
  task :stop, :roles => lambda { fetch(:sidekiq_role) }, :on_no_matching_servers => :continue do
  end

  desc 'Start sidekiq'
  task :start, :roles => lambda { fetch(:sidekiq_role) }, :on_no_matching_servers => :continue do
  end

  desc 'Restart sidekiq'
  task :restart, :roles => lambda { fetch(:sidekiq_role) }, :on_no_matching_servers => :continue do
  end    
end

When I need to restart one of my Sidekiq instances, I can just go to my terminal and execute the following:

$ bundle exec cap gandalf sidekiq:restart
$ bundle exec cap gollum sidekiq:stop 

It's made Sidekiq management quite painless for our team and thought it would be worth sharing in the event something similar could help you out.

Answer 2

You create two services:

cp sidekiq.conf /etc/init/cashyy.conf
cp sidekiq.conf /etc/init/glimee.conf

Edit each as necessary. sudo start cashyy , sudo stop glimee , etc. Now you'll have two completely separate Sidekiq processes running.