应用中的大插入量-性能

Question

I am having to create a custom tool in .NET to data migrate and cleanse an existing system. And it also does some other things.

I have literally a table of around 2000 users and then for each of those users, they can have anything from 0 to 9,000 "customer accounts"

then for each of these users and their accounts, they will need to insert a menu system into another table for that user and account.

So I am having to go through all these objects created within the app, and execute an insert statement. The problem is, the performance is horrible. It took 3.5 hours to complete the inserts and each insert took around 3 seconds to complete.... pretty slow!

With the inserts, I am making sure that I dont insert duplicate data/make sure there isnt any existing duplicate data thus my insert statment:

 IF NOT EXISTS (SELECT ID FROM UserWebAccessLevel WHERE UserID = '{0}' AND CustomerID = '{1}' AND MenuID = {2}) BEGIN INSERT INTO [WebAccessLevel] (UserID, CustomerID, MenuID) VALUES ('{0}', '{1}', {2}) END 

doing this for each user, for their customers and for each menu item found within the app logic.... yeh, takes a long time.

im wondering how I can better enhance the performance of the insert statement?

Thank you

Answer 1

Without an index on your table, the SELECT ID FROM... could be slowing things down, as SQL Server has to look at each record one by one and compare the IDs in the WHERE clause. By adding an index containing the three IDs, SQL Server will be able to (more-or-less) instantly locate an existing record when a WHERE clause specifies the three IDs.

Assuming you are using SQL Server Management Studio:-

• Right-click the table then click "Design".
• Right-click where the list of columns has appeared, then click "Indexes/Keys".
• in the next dialog that appears, click the Add button
• In the list of properties on the rh side, find the one called "Columns" and click the box to the right of it. When you see the ellipses button ("...") click that.
• In the next dialog that appears, add your three ID columns.

An index doesn't have to be unique. In the "Indexes/Keys" dialog you get the option to specify whether it should be unique or not.

The only other thing I can suggest checking is in your C#/VB code that executes the IF NOT EXISTS... SQL - make sure you aren't opening and closing the connection each time. That will really slow things down!