Rails 4.2 ActiveJob using Sidekiq jobs fail and then successfully retry on Heroku

Question

Actual Result:

We are getting failed mailer jobs showing up on sidekiq web UI ( using sidekiq-failures gem ) which get successfully retried after about a minute.

Failure Error:

ActiveRecord::RecordNotFound: Couldn't find User with 'id'=234

Expected:

We expect the sidekiq worker to succeed right away and not fail and later get (albeit successfully) retried.

Setup: Heroku Cedar-14, Rails 4.2, Puma 2.11.2, Sidekiq 3.3.4, Sidekiq-failures 0.4.4

config/initializers/sidekiq.rb:

Sidekiq.configure_server do |config|
  database_url = ENV['DATABASE_URL']
  if database_url
    ENV['DATABASE_URL'] = "#{database_url}?pool=15"
    ActiveRecord::Base.establish_connection
  end
end

config/puma.rb:

workers Integer(ENV['WEB_CONCURRENCY'] || 2)
threads_count = Integer(ENV['MAX_THREADS'] || 5)
threads threads_count, threads_count

preload_app!

rackup      DefaultRackup
port        ENV['PORT']     || 3000
environment ENV['RACK_ENV'] || 'development'

on_worker_boot do
  ActiveRecord::Base.establish_connection

end

config/sidekiq.yml

---
:concurrency: 25
:queues:
  - default
  - mailers
  - [low, 1]
  - [med, 2]
  - [high, 3]

The email is triggered from a service run in the controller action:

app/services/user_events.rb

@user = User.find(234)
UserMailer.send_email(@user.id).deliver_later

app/mailers/user_mailer.rb

  def send_email(user_id)
    @user = User.find(user_id)

    # mandrill_headers
    headers['X-MC-Track'] = "opens, clicks_all"
    headers['X-MC-Tags'] = ['user-event']

    mail(to: @user.email, subject: "Hi", css: ["email"])
  end

Answer 1

Is @user a newly-created record? If so, my hunch is that you are enqueuing the sidekiq email job before the current database transaction has been committed.

In such a scenario, the new user exists in your Rails transaction, but other processes connected to the database can't see it yet because the new user record has not been committed to the database. When sidekiq runs the email job (it is running in a separate process from your Rails app), it tries to load the user and gets a RecordNotFound .

The solution is to enqueue the sidekiq job outside (after) the database transaction. If you are using ActiveRecord callbacks, that means you should use after_commit instead of after_create .

From the sidekiq FAQ: https://github.com/mperham/sidekiq/wiki/FAQ#why-am-i-seeing-a-lot-of-cant-find-modelname-with-id12345-errors-with-sidekiq

Answer 2

Problem solved. It turned out to be that a heroku rollback reverted an environment variable.