Synchronize SQL Server databases

Question

I have a new idea and question about that I would like to ask you.

We have a CRM application on-premise / in house. We use that application kind of 24X7. We also do billing and payroll on the same CRM database which is OLTP and also same thing with SSRS reports.

It looks like whenever we do operation in front end which does inserts and updates to couple of entities at the same time, our application gets frozen until that process finishes. eg extracting payroll for 500 employees for their activities during last 2 weeks. Basically it summarize total working hours pulls that numbers from database and writes/updates that record where it says extract has been accomplished. so for 500 employees we are looking at around 40K-50K rows for Insert/Select/Update statements together.

Nobody can do anything while this process runs! We are considering the following options to take care of this issue.

1. Running this process in off-hours
2. OR make a copy of DB of Dyna. CRM and do this operations(extracting thousands of records and running multiple reports) on copy.

My questions are:

1. how to create first of all copy and where to create it (best practices)?
2. How to make it synchronize in real-time.
3. if we do select statement operation in copy DB than it's OK, but if we do any insert/update on copy how to reflect that on actual live db? , in short how to make sure both original and copy DB are synchronize to each other in real time.

I know I asked too many questions, but being SQL person, stepping into CRM team and providing suggestion, you know what I am trying to say.

Thanks folks for your any suggestion in advance.

Answer 1

Well to answer your question in regards to the live "copy" of a database a good solution is an alwayson availability group.

https://blogs.technet.microsoft.com/canitpro/2013/08/19/step-by-step-creating-a-sql-server-2012-alwayson-availability-group/

Though I dont think that is what you are going to want in this situation. Alwayson availability groups are typically for database instances that require very low failure time frames. For example: If the primary DB server goes down in the cluster it fails over to a secondary in a second or two at the most and the end users only notice a slight hiccup for a second.

What I think you would find better is to look at those insert statements that are hitting your database server and seeing why they are preventing you from pulling data. If they are truly locking the table maybe changing a large amount of your reads to "nolock" reads might help remedy your situation.

It would also be helpful to know what kind of resources you have allocated and also if you have proper indexing on the core tables for your DB. If you dont have proper indexing then a lot of the queries can take longer then normal causing the locking your seeing.

Finally I would recommend table partitioning if the tables you are pulling against are to large. This can help with a lot of disk speed issues potentially and also help optimize your querys if you partition by time segment (ie make a new partition every X months so when a query pulls from one time segment they only pull from that one data file).

https://msdn.microsoft.com/en-us/library/ms190787.aspx

I would say you need to focus on efficiency more then a "copy database" as your volumes arent very high to be needing anything like that from the sounds of it. I currently have a sql server transaction database running with 10 million+ inserts on it a day and I still have live reports hit against it. You just need the resources and proper indexing to accommodate.