Calling worker inside another worker with sidekiq,

Question

I have problem with calling worker inside another worker with sidekiq. I have a two workers that looks like this:

class UserExportWorker
  include Sidekiq::Worker

  sidekiq_options queue: :users, unique: true, retry: false

  def perform(user_id)
    # code that exports user to external service via API
    UserExportStatusWorker.perform_in(1.minute, user_export_id, user_id)
  end
end

class UserExportStatusWorker
  include Sidekiq::Worker

  sidekiq_options queue: :users, unique: true, retry: false

  def perform(user_export_id, user_id)
    # code that check if exporting user to external service via API is finished
    if export.completed?
      user.update(status: 'exported')
    else
      UserExportStatusWorker.perform_in(1.minute, user_export_id, user_id)
    end
  end
end

UserExportWorker exports user to some external service via API and run UserExportStatusWorker that will check if export in external API is finished and, if its not then it will launch the same worker again, when export will finish it will update user state column in database. The problem is that UserExportStatusWorker is not updating user in database even if export to external API is successfull. But when I change my code to this:

class UserExportWorker
  include Sidekiq::Worker

  sidekiq_options queue: :users, unique: true, retry: false

  def perform(user_id)
    # code that exports user to external service via API
    UserExportStatusWorker.new.perform(user_export_id, user_id)
  end
end

class UserExportStatusWorker
  include Sidekiq::Worker

  sidekiq_options queue: :users, unique: true, retry: false

  def perform(user_export_id, user_id)
    # code that check if exporting user to external service via API is finished
    if export.completed?
      user.update(status: 'exported')
    else
      UserExportStatusWorker.new.perform(user_export_id, user_id)
    end
  end
end

I'm completely out of ideas... Why this not works?

Answer 1

Ideally to cause a restart on a Worker you should be raising an Error, and letting sidekiq retry it, as described in the docs .

So in your case, in your UserExportWorker class you could have a method to check if the exporting ran successfully, and if it didn't, raise a custom ExportNotSuccessfulError