Rails Sidekiq Queue size not working

Question

I have a Rails app with 2 jobs (ImportCsvJob and ProcessCsvJob). So that I can visibly hint in the app that there are still jobs in the queue I have this helper method (inside application_helper.rb):

module ApplicationHelper
   def queued_job_count
        Sidekiq::Queue.new.size
    end
end

Then I use it on my index controller which is then passed to the view for processing and giving the visual hint in the app.

  def index
    @still_have_jobs = !queued_job_count.zero?
  end

However, this works when I still had 1 background Job (ImportCsvJob), but when I added the (ProcessCsvJob) it does not work anymore.

import_csv_job.rb

require 'open-uri'

class ImportCsvJob < ActiveJob::Base
  queue_as :default

  def perform(csv_record)
    csv_record[:object_changes] = ApplicationController.helpers.generate_hash(csv_record[:object_changes])
    ObjectRecord.create(csv_record)
  end
end

process_csv_job.rb

class ProcessCsvJob < ActiveJob::Base
  queue_as :default

  def perform(csv_path)
    csv_file = open(csv_path,'rb:UTF-8')

    options = {
        row_sep: :auto, col_sep: ",", 
        user_provided_headers: [:object_id, :object_type, :timestamp, :object_changes], 
        remove_empty_values: true, 
        headers_in_file: true
    }

    SmarterCSV.process(csv_file, options) do |array|
        ImportCsvJob.perform_later(array.first)
    end
  end

end

and lastly, in the model where this is called:

ProcessCsvJob.perform_later(gdrive.uploaded_file_link)

When I try to debug in Rails console using Sidekiq::Queue.new.size, it still gives out 0.

Running:

redis-server
bundle exec sidekiq

Answer 1

A job that is executing is not enqueued anymore. The Sidekiq process has already popped it off the queue and is executing it. The queue is empty but the job is not finished yet.

Answer 2

So, basically I added a monitoring for sidekiq using the web interface to see what was happening:

And as I inspected, there were no enqueued tasks nor scheduled since most of the job is set to perform almost immediately (on parallel).

Thus here's my solution to know if the count of busy jobs:

module ApplicationHelper
    def queued_job_count
        Sidekiq::ProcessSet.new.first['busy']
    end
end

and then on the index:

  def index
    @still_have_jobs = !queued_job_count.zero?
  end

it works! :)