Sporadic Execution Times for Query in SQL Server 2008

Question

I have been running some speed tests on a query where I insert 10,000 records into a table that has millions (over 24mil) of records. The query (below) will not insert duplicate records.

MERGE INTO [dbo].[tbl1] AS tbl
USING (SELECT col2,col3, max(col4) col4, max(col5) col5, max(col6) col6 FROM #tmp group by col2, col3) AS src
ON (tbl.col2 = src.col2 AND tbl.col3 = src.col3)
WHEN NOT MATCHED THEN 
INSERT (col2,col3,col4,col5,col6)
VALUES (src.col2,src.col3,src.col4,src.col5,src.col6);

The execution times of the above query are sporadic; ranging anywhere from 0:02 seconds to 2:00 minutes.

I am running these tests within SQL Server Studio via a script that will create the 10,000 rows of data (into the #tmp table), then the MERGE query above is fired. The point being, the same exact script is executing for each test that I run.

The execution times bounce around from seconds to minutes as in:

• Test #1: 0:10 seconds
• Test #2: 1:13 minutes
• Test #3: 0:02 seconds
• Test #4: 1:56 minutes
• Test #5: 0:05 seconds
• Test #6: 1:22 minutes

One metric that I find interesting is that the seconds/minutes alternating sequence is relatively consistent - ie every other test the results are in seconds.

Can you give me any clues as to what may be causing this query to have such sporadic execution times?

Answer 1

I wish I could say what the cause of the sporadic execution times was, but I can say what I did to work around the problem...

I created a new database and target table and added 25 million records to the target table. Then I ran my original tests on the new database/table by repeatedly inserting 10k records into the target table. The results were consistent execution times of aprox 0:07 seconds (for each 10k insert).

For kicks I did the exact same testing on a machine that has twice as much CPU/Memory than my dev laptop. The results were consistent execution times of 0:00 seconds (It's time for a new dev machine;))

I dislike not discovering the cause to the problem, but in this case I'm going to have to call it good and move on. Hopefully, someday, a StackO die-hard can update this question with a good answer.