ASP.NET Core Distributed Redis Cache: Disconnect

Question

I'am using Redis cache as distributed cache in ASP.NET app. It works until Redis server becomes unavailable and the question is: How to properly handle disconnection issues?

Redis is configured this way (Startup.cs):

services.AddDistributedRedisCache(...)

Option AbortOnConnectFail is set to false

Injected in service via constructor:

...
private IDistributedCache _cache

public MyService(IDistributedCache cache)
{
  _cache = cache;
}

When Redis is down the following code throws an exception (StackExchange.Redis.RedisConnectionException: SocketFailure on 127.0.0.1:6379/Subscription...):

var val = await _cache.GetAsync(key, cancellationToken);

I don't think that using reflection to inspect a connection state inside _cache object is a good way. So are there any 'right' options to handle it?

Answer 1

Maybe you can check Polly Project . It has Retry/WaitAndRetry/RetryForever and Circuit Breakers that can be handy. So you can catch that RedisConnectionException And then retry or fallback to other method.

You have Plugin for Microsoft DistributedCache Provider.

Check it out.

Answer 2

First of all, why is your Redis server becoming unavailable? And for how long? You should minimize these kinds of situations. Do you use Redis as a service from AWS ie ElasticCache? If so you can configure it to promote a new Redis slave /read-replice server to become a master if the first master fails.

To improve fault tolerance and reduce write downtime, enable Multi-AZ with Automatic Failover for your Redis (cluster mode disabled) cluster with replicas. For more information, see Minimizing downtime in ElastiCache for Redis with Multi-AZ.

https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/AutoFailover.html

Apart from that, a fallback solution to an unresponsive Redis server would be just to retrieve the objects/entities that your a caching in Redis from the database if the Redis server is down. You can retry the Redis call two times with 5 seconds between each retry and if the server is still down you should just query the database. This would result in a performance hit but it is a better solution than throwing an error.

T val = null;
int retryCount = 0;

do 
{
      try 
      {
          val = await _cache.GetAsync(key, cancellationToken);
      }
      catch(Exception ex) 
      {  
         retryCount++;
         Thread.Sleep(retryCount * 2000)
      }
 }
 while(retryCount < 3 && val == null);
 
 if (val == null) 
 {
    var = call to database
 }