在数据库中保存非常繁忙的计数器的最佳实践

Question

I am developing a very busy web service which is supposed to get thousands of requests per second. I want that each request will update a counter field. How would i do that? Saving the counter in db is very important so I will not lose the information in case the server dies. I have tried the following code, but this will be a huge bottle neck for thousands requests per second. How would you do that?

public void Upload(int organizationId)
{
  try
  {
    lock (UpadateLock)
    {
       using (var db = new DbContext())
       {
         Counter counter = db.Counters.Where(c => c.OrganizationId == organizationId).FirstOrDefault();
         counter.count++;
         db.SaveChanges();

       }
     }
   }
   catch (Exception ex)
   {

   }
}

Answer 1

If you absolutely cannot lose data when the server dies then you must write to a persistent store for each increment.

The question is just what store to use.

You can certainly do it with SQL. A simple query like this is not too expensive. You can benchmark this to see if the overhead is acceptable. Measure CPU usage on web server and SQL server. Also measure disk usage and transaction log size.

Redis might turn out to be a really good database for this. It is known to be fast and it supports server-side increments.

You can also scale out this workload by writing increments to multiple servers (randomly picking one).

You can use any other data store as well. If it does not support incrementing natively you can instead insert new rows. Then, have a background process that periodically aggregates those rows so that they don't keep accumulating.