C# Transaction Inside loop

Question

To explain my scenario, I have taken 2 tables Department and Employee and it's structure will be like below,

Department      
DepartmentId    int IDENTITY
DepartmentName  varchar(2500    
Description varchar(500)    

Employee        
EmployeeId  int IDENTITY
EmployeeName    varchar(250)    
Age int 
DepartmentId    int 
IsActive    bit 

We used to get the employee and the associated departments in a flat file and the volume will be between 100 - 500 records. So, we extract the data, validate it and all valid records are stored in a list (List of FileRow).

Finally, the valid records from the list are iterated and on each iteration, we do the following,

Department detail is inserted
Departmentid value is retrieved
DepartmentId is assigned to Employee object and inserted into Employee Table.

As of now, we have started the transaction (TransactionScope) before the iteration (For loop) and the transaction is completed, after the iteration. all rows are unrelated so they can inserted individually as well.

So, is this correct or do we need to create the transaction on each insertion? means inside the loop? please advice which will be give better performance and also will not affect other systems which utilize the same database?

Logic will be,

Start Transaction
Iterate each valid FileRow from the list
insert into department
SaveChanges() method is called, Department ID will be generated
Take the DepartmentId value
Insert into Employee
Iteration ends
Transaction Complete

Thanks, Prakash.

Answer 1

So, is this correct or do we need to create the transaction on each insertion?

That is not a technical question. Both will work, but both will have different behaviours: A transaction is a package that will either all work or none work. So what do you want to happen if a record is faulty in your files? Do you want all to fail? Or do you want to insert all but the faulty record? That would be your first hint what is the right answer for you in your specific situation .

With the next loop for the next program, the answer might be different.

Answer 2

As I commented, it depends on what you want to achieve. If a single invalid record indicates that the entire set of records cannot be trusted then your design is fine. But, if you want to treat the validity of each record as unrelated to the quality of the entire set then you need to have a more granular approach with a transaction per record.

However, if you are simply bothered about performance, then there a few issues to consider. Having a single transaction could, depending on the isolation level, prevent other users from reading or updating data from the Department and Employee tables for the duration of the transaction.

Having a transaction per record would have the overhead of, in your case, creating 100-500 transactions. That doesn't sound much to me and would not impede other users noticeably.

Since you don't seem to require the entire set of records to be valid I would go for the lower level transaction per record and monitor the performance, although I doubt it would be an issue.