Sidekiq 抛出 Redis::CommandError

Question

I have this redis container I use for Sidekiq/Rails and also set up rabbitmq a few days ago on the same docker host. Ever since I keep getting these errors:

UNBLOCKED force unblock from blocking operation, instance state changed (master -> slave?)

redis/client.rb in call at line 126
    def call(command)
      reply = process([command]) { read }
      raise reply if reply.is_a?(CommandError)

      if block_given?
        yield reply

redis/client.rb in block in call_with_timeout at line 219
redis/client.rb in with_socket_timeout at line 292
redis/client.rb in call_with_timeout at line 218
redis.rb in block in _bpop at line 1186
redis.rb in block in synchronize at line 52
monitor.rb in mon_synchronize at line 226
redis.rb in synchronize at line 52
redis.rb in _bpop at line 1183
redis.rb in brpop at line 1228
redis/namespace.rb in call_with_namespace at line 469
redis/namespace.rb in block (2 levels) in <class:Namespace> at line 349
sidekiq/fetch.rb in block in retrieve_work at line 37
sidekiq.rb in block in redis at line 97
connection_pool.rb in block (2 levels) in with at line 65
connection_pool.rb in handle_interrupt at line 64
connection_pool.rb in block in with at line 64
connection_pool.rb in handle_interrupt at line 61
connection_pool.rb in with at line 61
sidekiq.rb in redis at line 94
sidekiq/fetch.rb in retrieve_work at line 37
sidekiq/processor.rb in get_one at line 83
sidekiq/processor.rb in fetch at line 95
sidekiq/processor.rb in process_one at line 77
sidekiq/processor.rb in run at line 68
sidekiq/util.rb in watchdog at line 15
sidekiq/util.rb in block in safe_thread at line 24 

The redis logs show nothing relevant, just the "save database" stuff.

Have I done something evil getting them both set up within the same host?

Answer 1

Turns out someone removed a firewall rule so the redis port was exposed to datacenter-wide VLAN.

And since the datacenter where that machine lives have promised 3 years ago they will be releasing a feature where each customer/account can have their own separate VLAN, they have yet to and some of their customers seem to be up to no good.

The error I was seeing was someone probing ports and trying all sorts of remote exploits.

After adding the firewall rule back up, the error disappeared.