Using Sidekiq for Active Job and getting ActiveJob::DeserializationError

Question

I'm trying to use Sidekiq to run the below job.

The job performs fine when not queued (perform_now) but fails when called as (perform_later), which uses Sidekiq.

AddEmployeesToRoomJob.perform_now room  ## works fine
AddEmployeesToRoomJob.perform_later room  ## breaks in Sidekiq

Error:

AddEmployeesToRoomJob JID-da24b13f405b1ece1212bbd5 INFO: fail: 0.003     sec
2016-08-20T14:57:16.645Z 19456 TID-owmym5fbk WARN:     {"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped"    :"AddEmployeesToRoomJob","queue":"default","args":    [{"job_class":"AddEmployeesToRoomJob","job_id":"0ba5bd30-e281-49a7-a93f-    6e50445183ac","queue_name":"default","priority":null,"arguments":    [{"_aj_globalid":"gid://dragonfly/Room/1"}],"locale":"en"}],"retry":true,    "jid":"da24b13f405b1ece1212bbd5","created_at":1471704675.739077,"enqueued    _at":1471705036.6406531,"error_message":"Error while trying to     deserialize arguments: Couldn't find Room with     'id'=1","error_class":"ActiveJob::DeserializationError","failed_at":14717    04675.946183,"retry_count":4,"retried_at":1471705036.644416}
2016-08-20T14:57:16.645Z 19456 TID-owmym5fbk WARN:     ActiveJob::DeserializationError: Error while trying to deserialize     arguments: Couldn't find Room with 'id'=1
2016-08-20T14:57:16.645Z 19456 TID-owmym5fbk WARN:     /Users/tamlyn/.rvm/gems/ruby-2.2.3/gems/activerecord-    5.0.0.1/lib/active_record/relation/finder_methods.rb:357:in     `raise_record_not_found_exception!'

My Job class AddEmployeesToRoomJob < ApplicationJob queue_as :default

  def perform(room)
    employees = Employee.all
    if employees.length > 0
      employees.each do |employee|
        UserRoom.create(user: employee, room: room)
      end
    end
  end
end

My Thoughts I don't understand why it can't find the room which I'm passing into the perform method. It's as though it somehow loses that variable in the queueifying / JSONifying of the job?

The Sidekiq docs say

"Unfortunately this means that if the [Room] record is deleted after the job is enqueued but before the perform method is called, exception handling is different."

They suggest a workaround but I don't see how that would help me:

rescue_from ActiveJob::DeserializationError do |exception|
    # handle a deleted user record
end

Thanks in advance for any help!

Answer 1

I don't think it is a good idea to pass the Room object into a Sidekiq worker. I've always passed the primary key for a database object and then re-queried. Try this.

AddEmployeesToRoomJob.perform_later room.id

def perform(room_id)
  room = Room.find(room_id)
  employees = Employee.all
    if employees.length > 0
      employees.each do |employee|
        UserRoom.create(user: employee, room: room)
      end
    end
  end
end

Answer 2

Stumbled upon this myself and found the method discard_on very useful. Mostly it is very unnecessary to do jobs on deleted or never created records.

Example:

class ApplicationJob < ActiveJob::Base
  discard_on ActiveJob::DeserializationError do |job, error|
    Rails.logger.error("Skipping job because of ActiveJob::DeserializationError (#{error.message})")
  end
end

Answer 3

You can pass a model to a job if you can ensure that this model has been commited. It's a classic error to enqueue jobs before commit the current transaction.

You can find more information here: https://github.com/mperham/sidekiq/wiki/FAQ#why-am-i-seeing-a-lot-of-cant-find-modelname-with-id12345-errors-with-sidekiq

In our project we coded a model concern to be able to add dynamic after commit code blocks.

module AfterCommitOnce
  extend ActiveSupport::Concern

  included do
    after_commit :execute_after_commit_handlers
  end

  def after_commit_once(&block)
    @after_commits = @after_commits || []
    @after_commits << block
  end

  private

  def execute_after_commit_handlers
    @after_commits = @after_commits || []
    @after_commits.each do |ac|
      ac.call
    end
    @after_commits = []
  end
end

Adding this concern to you Employee model to you can specify which job you want to execute after commit all changes to you database with:

...
employee.after_commit_once do
   AddEmployeesToRoomJob.perform_later
end
employee.save!

Answer 4

It's not advisable to send Ruby object as parameter to sidekick worker. You can send Id as parameter and initialise the object inside perform method. If want to send object then you can dump Ruby object to any other format like Json/ Binary / yml as follows.

object.to_json 

Or

Marshal.dump(object)

And before using the objects inside perform method you can de-serialize the object to Ruby object as follows.

JSON.parse(serialized_object)

Or

Marshal.load(serialized_object)

These are solutions for your problem, but not ideal solution.