Sidekiq failing a lot before succeeding

Question

I've set up sidekiq to run after_commit, but it fails (close to) 100% of the time with ActiveRecord::RecordNotFound: Couldn't find User with id=42635 .

For the longest time I thought it was connected to the wrong redis database, but with retry: true turned on I've found that it eventually succeeds after around 10 minutes.

Its very weird, because I can see our users in the admin panel with the id that failed, but sidekiq will still fail for a while then eventually a retry will work. I have no idea what's causing this, I'd like to get it to succeed on the first try.

Edit: Using AWS, example sidekiq call and worker below:

In UserObserver:

  def after_commit(user)
    if user.created_at == user.updated_at
      @user = user
      identify_and_track
    end

...

def identify_and_track
  IdentifyAndTrackUserWorker.perform_async(@user.id)
end

Sidekiq worker:

class IdentifyAndTrackUserWorker
  include Sidekiq::Worker
  sidekiq_options retry: true

  def perform(user_id)
    @user = User.find user_id
    Analytics.identify(
      user_id: user_id,
      traits: { email:      @user.email,
                first_name: @user.first_name,
                last_name:  @user.last_name
              }
    )
  end
end

Answer 1

I have had problems very similar to this one and have come to find out that Sidekiq, in it's speedy nature, is executing the job before Rails is able to create the object. To prevent sidekiq failure, it was helper to give Rails the time to create the object.

Use IdentifyAndTrackUserWorker.perform_in(1.minute, @user.id) rather than IdentifyAndTrackUserWorker.perform_async(@user.id) .