Heroku + Sidekiq + Redis + Puma configuration issue

Question

I have an application in Heroku that is running with sidekiq + Redis + puma.

Heroku configurations:

Redis Total plan Connection limit: 200 (using Heroku Redis)

puma.rb

threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
workers ENV.fetch("WEB_CONCURRENCY") { 2 }

sidekiq.rb

require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { size: 1 }
end

Sidekiq.configure_server do |config|
  config.redis = { size: 4 }
end

sidekiq.yml

production:
  :concurrency: 2

redis.rb

Redis.current = Redis.new(url: ENV[ENV['REDIS_PROVIDER'] || 'REDIS_URL'])

This above configuration is working fine, but I couldn't make use of maximum connections as per this above configuration currently that uses 34 connections

Also when I change this above configuration that is not working getting these errors:

could not obtain a connection from the pool within 5.000 seconds (waited 5.002 
seconds); all pooled connections were in use

I am following this calculator

I have also used the REDIS_URL instead of size,

Sidekiq.configure_client do |config|
  config.redis = { url: ENV['REDIS_URL'] || 'redis://localhost:6379/1' }
end

Sidekiq.configure_server do |config|
  config.redis = { url: ENV['REDIS_URL'] || 'redis://localhost:6379/1' }
end

that time also we are getting this error: could not obtain a connection from the pool within 5.000 seconds (waited 5.007 seconds); all pooled connections were in use

Full Backtrace:

could not obtain a connection from the pool within 5.000 seconds (waited 5.002 seconds); all pooled connections were in use..["/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:202:in `block in wait_poll'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:193:in `loop'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:193:in `wait_poll'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:154:in `internal_poll'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:278:in `internal_poll'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:148:in `block in poll'", "/app/vendor/ruby-2.4.4/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:158:in `synchronize'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:148:in `poll'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:747:in `acquire_connection'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:500:in `checkout'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:374:in `connection'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:931:in `retrieve_connection'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_handling.rb:116:in `retrieve_connection'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/connection_handling.rb:88:in `connection'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/associations/join_dependency.rb:96:in `initialize'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/relation/query_methods.rb:1016:in `new'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/relation/query_methods.rb:1016:in `build_join_query'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/relation/query_methods.rb:1003:in `build_joins'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/relation/query_methods.rb:942:in `build_arel'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/relation/query_methods.rb:918:in `arel'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/relation.rb:678:in `exec_queries'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/relation.rb:546:in `load'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/relation.rb:255:in `records'", "/app/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6/lib/active_record/relation/delegation.rb:39:in `each'", "/app/app/workers/pos/refresh_menu_worker.rb:19:in `perform'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:185:in `execute_job'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:167:in `block (2 levels) in process'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/middleware/chain.rb:133:in `invoke'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:166:in `block in process'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/job_retry.rb:98:in `local'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/rails.rb:42:in `block in call'", "/app/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6/lib/active_support/execution_wrapper.rb:85:in `wrap'", "/app/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6/lib/active_support/reloader.rb:68:in `block in wrap'", "/app/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6/lib/active_support/execution_wrapper.rb:85:in `wrap'", "/app/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6/lib/active_support/reloader.rb:67:in `wrap'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/rails.rb:41:in `call'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:217:in `stats'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/job_logger.rb:8:in `call'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/job_retry.rb:73:in `global'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:125:in `block in dispatch'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/logging.rb:48:in `with_context'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/logging.rb:42:in `with_job_hash_context'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:124:in `dispatch'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:165:in `process'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:83:in `process_one'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/processor.rb:71:in `run'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/util.rb:16:in `watchdog'", "/app/vendor/bundle/ruby/2.4.0/gems/sidekiq-5.2.1/lib/sidekiq/util.rb:25:in `block in safe_thread'"]

Can anyone suggest the configuration to accommodate maximum connection?

Answer 1

web dynos: 4 dynos * 3 client connections per process = 12

workers: 6 workers * (5 redis connections per worker (by default) + concurrency set to 10) = 90

You can get a very good read over here https://medium.com/appaloosa-store-engineering/you-should-revise-your-redis-max-connections-setting-8136f063c916