将mssql表事务记录到Redis

Question

I have mssql database and tons of tables have update, delete triggers. All tables are inserting the changes into one Log table inside same database. Also we have sql job which moves logs into another database which contains only log table. I want to make it faster this operation with NoSQL database like this:

MSSQL->
  MainDB->
    update/delete->
      triggers 
           collect before,update rows into xml 
           and insert as JSON into Redis db

I don't know this is faster than this:

MSSQL->
  MainDB->
    update/delete->
      triggers 
          collect before,update rows into xml 
          and insert into log table in the same db

If NoSQL would be faster than itself, how should I call Redis api from MSSQL Trigger?

Answer 1

This depends on what you want to use your logs for, but a few pieces of advice:

1. It sounds like you're going to have a lot of logs. Among many other reasons not to use Redis for this, it is an in-memory datastore, and you are going to be wasting a ton of memory just to store logs, which I'm guessing you really don't need fast access to. Just fast writes. You can get fast writes much more cheaply.

2. There are lots of great distributed logging frameworks that will do what you want ( Kafka , Fluentd , etc...). FluentD is really easy to use and will let you configure your logging to run in a distributed manner that will be fast and disconnected from your database. It has many connectors that will let you stream your logs to various stores (MongoDB, MySQL, flat files, S3 etc...). Your choice of datastore should depend on what you eventually want to do with your logs.

3. Do you have to call the log from an MsSQL trigger? It's an unnecessary hit on your database, and you could easily call the log from your application instead. This will make the whole thing distributed per app box and take the load off your db.