使用并发插入避免 SQL 中的死锁

Question

I have a process (Process A) which keeps adding records to a SQL table (Table A) (Direct inserts using stored procedure). It is a continuous process which reads the requests and writes to a table. There is no pattern to how the requests come. The max requests per day is around 100K.

Once the requests come in, I need to do some processing on those requests. These are currently done in user desktops (due to licensing issues). The way I am currently doing is having an executable (Process B) run on each user and as and when requests come in to the table, this process reads and does some work and writes to the same table. So Table is read/written by multiple processes. The process B has the following logic

• Get records that have not been processed by another user and is not being currently processed by another user

  • Lock the records for this run by marking a flag isProcessing (C# LINQ through SP). This is a single SQL transaction ie lock records and get them for processing is wrapped in a transaction

• process the records. This is where the calculation occurs. No db work here.

• insert/update records in table A (C# LINQ through db.submitchanges). This is where the deadlock occurs. This is a separate SQL transaction.

Occasionally, I see deadlocks when writing to the table. This SQL Server 2008 (with isolation level Read committed). Access to SQL is done by both Stored procedures and direct C# Linq Queries. Question is how to avoid the deadlocks. Is there a better overall architecture ? Maybe, instead of all these child processes writing to the table independently, I should send them to a service which queues them up and writes to the table ?. I know it is tough to answer without having all the code (just too many to show) but hopefully I have explained it and I will happy to answer any specific questions.

This is a representative table structure.

    CREATE TABLE [dbo].[tbl_data](
[tbl_id] [nvarchar](50) NOT NULL,
[xml_data] [xml] NULL, -- where output will be stored
[error_message] [nvarchar](250) NULL,
[last_processed_date] [datetime] NULL,
[last_processed_by] [nvarchar](50) NULL,
[processing_id] [uniqueidentifier] NULL,
[processing_start_date] [datetime] NULL,
[create_date] [datetime] NOT NULL,
[processing_user] [nvarchar](50) NULL,
    CONSTRAINT [PK_tbl_data] PRIMARY KEY CLUSTERED 
    (
[tbl_id] ASC,
[create_date] ASC
    ) ON [PRIMARY]

This is the proc that gets the data for processing.

    begin tran
            -- clear processing records that have been running for more than 6 minutes... they need to be reprocessed...
    update tbl_data set processing_id = null, processing_start_date = null
    where DATEDIFF(MINUTE, processing_start_date, GETDATE()) >=6

    DECLARE @myid uniqueidentifier = NEWID();

    declare @user_count int

    -- The literal number below is the max any user can process. The last_processed_by and last_processed_date are updated when a record has been processed
    select @user_count = 5000 - count(*) from  tbl_data where last_processed_by = @user_name and  DATEDIFF(dd, last_processed_date, GETDATE()) = 0

    IF (@user_count > 1000) 
        SET @user_count = 1000 -- no more than 1000 requests in each batch.

    if (@user_count < 0) 
        set @user_count = 0



    --mark the records as being processed
    update tbl_data set processing_id = @myid, processing_start_date = GETDATE(), processing_user = @user_name from tbl_data t1 join
    (
        select top (@user_count) tbl_id from tbl_data
        where 
            [enabled] = 1 and processing_id is null 
        and isnull(DATEDIFF(dd, last_processed_date, GETDATE()), 1) > 0 
        and isnull(DATEDIFF(dd, create_date, GETDATE()), 1) = 0 
    ) t2 on t1.tbl_id = t2.tbl_id

    -- get the records that have been marked
    select tbl_id from tbl_data where processing_id = @myid 

    commit tran

Answer 1

My guess is you are deadlocking on pages as concurrent updates are attempted.

With the nature of the updates and inserts (a sliding timeframe window based on getdate), it looks like a good partitioning scheme is difficult to implement. Without it, I think your best option would be to implement an application level lock (the sql equivalent of a mutex) using sp_getapplock http://msdn.microsoft.com/en-us/library/ms189823(v=sql.100).aspx

Answer 2

I lack the time right now to analyze your workload and find a true fix. So I'm going to add a different kind of answer: You can safely retry deadlocking transactions. This problem can be fixed by just re-running the entire transactions. Maybe a little delay needs to be inserted before a retry is attempted.

Be sure to rerun the entire transaction, though, including any control flow that happens in the application. In case of a retry the data that was already read might have changed.

If retries are rare this is not a performance problem. You should probably log when a retry happened.