How can I ensure Sidekiq background jobs run against the same database they were initiated from

Question

I have been implementing horizontal database sharding in Rails 6.1 using the new native multiple database connection switching. This has worked great in general, but I am trying to find the best way ensure that background jobs (via Sidekiq backed by a shared Redis DB) are run against the same database shard that they were initiated from.

eg I have two horizontal database shards, Database A and Database B. The job is enqueued from a web process while connected to database A (using a Rack middleware). When that job is run later in the Sidekiq process, I would like the job to be automatically run against Database A.

A brute-force way to do this would be to pass the current connected database as an argument into every job, and then, in each job, connect to the database specified in the argument using the functionality at https://guides.rubyonrails.org/active_record_multiple_databases.html .

If I was using an older solution to horizontal sharding like Apartment, the https://github.com/influitive/apartment-sidekiq gem would be exactly what I needed. However I cannot find any solutions on how to do something similar with Rails's new native functionality.

What is the best way to accomplish this?

Answer 1

This is impossible. Jobs can be created in Process A and executed in Process B. Connection sharing is impossible between processes.

Answer 2

You want to save the current shard using a client middleware and restore it in a server middleware. See the notes about ActiveSupport::CurrentAttributes and the Middleware wiki page .