Sidekiq jobs won't run in same time in different queues

Question

I have 2 Sidekiq workers:

Foo:

​# frozen_string_literal: true
  class FooWorker
    include Sidekiq::Worker
    sidekiq_options queue: :foo
    def perform
      loop do
        File.open(File.join(Rails.root, 'foo.txt'), 'w') { |file| file.write('FOO') }
      end
    end
  end

Bar:

# frozen_string_literal: true
  class BarWorker
    include Sidekiq::Worker
    sidekiq_options queue: :bar
    def perform
      loop do
        File.open(File.join(Rails.root, 'bar.txt'), 'w') { |file| file.write('BAR') }
      end
    end
  end

Which has pretty the same functionality, both runs on different queues and the yaml file looks like this:

---
:queues:
  - foo
  - bar

development:
  :concurrency: 5

The problem is, even both are running and showing in the Busy page of Sidekiq UI, only one of them will actually create a file and put contents in. Shouldn't Sidekiq be multi-threaded?

Update:

• this happens only on my machine
• i created a new project with rails new and same
• i cloned a colleague project and ran his sidekiq and is working!!!
• i used his sidekiq version, not working!

New Update:

• this happens also on my colleague machine if he clone my project
• if I run 2 jobs with a finite loop ( like 10 times do something with a sleep), first job will be executed and then the second, but after the second finishes and start again both will work on same time as expected -- everyone who cloned the project from: github.com/ArayB/sidekiq-test encountered the problem.

Answer 1

It's not an issue with Sidekiq. It's an issue somewhere in Ruby/MRI/Thread/GIL. Google for more info, but my understanding is that sometimes threads aren't real threads (see "green threads") so really just simulate threading. The important thing is that only one thread can execute at a time.

It's interesting that with only two threads the system isn't giving time to the second thread. No idea why, but it must realize it's mistake when you run it again.

Interestingly if you run your same app but instead fire off 10 TestWorkers (and tweak the output so you can tell the difference) sidekiq will run all 10 "at once".

10.times {|i| TestWorker.perform_async(i) }

Here is the tweaked worker. Be sure to flush the output cause that can also cause issues with threading and TTY buffering not reflecting reality.

class TestWorker
  include Sidekiq::Worker

      def perform(n)
        10.times do |i|
          puts "#{n} - #{i} - #{Time.current}"
          $stdout.flush
          sleep 1
        end
      end
    end

Some interesting links:

• https://en.wikipedia.org/wiki/Green_threads
• http://ruby-doc.org/core-2.4.1/Thread.html#method-c-pass
• https://github.com/ruby/ruby/blob/v2_4_1/thread.c
• Does ruby have real multithreading?