Block commit after Update/Insert for 10 million records being processed

Question

Requirements need an Update and Insert for 10 million records in Sql Server 2005. I have created a Stored Procedure using T-SQL. Do we need to do a commit after the process above , say 10000 records committed at at time. If so how do I write a commit statement for say every 10000 records. Any ideas would be appreciated.

Answer 1

In general, a way to do batch deletes/updates/inserts is to use a WHILE loop. The pattern I normally follow is:

WHILE 1=1
BEGIN
 INSERT INTO dbo.MyTable
 (field1, field2, field3...)
 SELECT TOP 100000 (field1, field2, field3...)
 FROM dbo.MySourceTableorQuery as S
 WHERE NOT EXISTS (
        SELECT * FROM dbo.SourceTableorQuery as S 
        WHERE s.PrimaryKey = MyTable.PrimaryKey)
 IF @@ROWCOUNT<100000 BREAK;
END

As to "whether you need to", that's impossible to answer without a lot of other information. What kind of query is it, what's the table structure like, how long does it take to run currently, how are you for disk space (you will really increase the size of tempdb if it's a single transaction), how wide is the table you are inserting to, etc?