SQL Server Update Queries Taking Too Long

Question

I have a SQL query that builds analysis tables using SELECT INTO statements from populated staging tables. This part goes pretty smooth, but when I try to append fields to the analysis tables, from each other or from the original staging tables, the UPDATE queries run for HOURS.

I feel this run time is unwarranted because the most records per table is 200K. I have a feeling that this is an issue with my log file management or the fact that I have no indexes on any of these tables. I have been reluctant to build indexes because I know if hastily done, they can actually cause more problems. There are no triggers on any of these tables.

I am trying to find an approach so that this query runs consistently. In the past, it has run under 1 hour (which is still probably too long) but today for instance it has taken 3+ hours.

I've already modified the log file to autogrow by 50% rather than 10%, to have an unlimited MAXSIZE, I've set the DB recovery mode to SIMPLE and I have shrunk the log file manually. None of this is helping.

I know that TABLE SCANS are a source of lag in large queries, so is the solution here to create indexes for all of the fields I use in my LEFT JOINs depicted below? These snippets are exact examples of where the query is lagging. I have transparency into this because I am following the execution of the script in the LIVE QUERY STATISTICS window.

UPDATE a
SET a.[Account ID] = b.[Account ID]
FROM SFAX.dbo.Leads as a
left join SFAX.dbo.Accts as b on a.[Company   Account] = b.[Account Name]
WHERE a.[Account ID] IS NULL;

-- Link by website2
UPDATE a
SET a.[Account ID] = b.[Account ID]
FROM SFAX.dbo.Leads as a
left join SFAX.dbo.Accts as b on a.url2 = b.Website2
WHERE a.[Account ID] IS NULL;
--
UPDATE a
SET a.[SFDC Account Name] = b.[Account Name]
FROM SFAX.dbo.Leads as a
left join SFAX.dbo.Accts as b on
a.[Account ID] = b.[Account ID]
WHERE a.[Account ID] IS NOT NULL;

UPDATE a
SET a.ConvertedFlag = b.[Lead ID]
FROM SFAX.dbo.WhoAnalysis1 as a 
LEFT JOIN SFAX.dbo.Contacts as b ON a.WhoId = b.[Contact ID];

Answer 1

Note: In the comments above you express doubt. This is easy to test -- break your script in half prior to updates -- run part one -- then run one update and time it. Then make indexes for that one update. Run it again. How many orders of magnitude faster is it?

Don't be scared of indexes if you work with SQL -- love your indexes. Create indexes for

SFAX.dbo.Leads.[Company   Account], [Account ID] 
SFAX.dbo.Accts.[Account Name]

SFAX.dbo.Leads.url2, [Account ID]
SFAX.dbo.Accts.Website2

SFAX.dbo.Leads.[Account ID]
SFAX.dbo.Accts.[Account ID]

SFAX.dbo.WhoAnalysis1.WhoID
SFAX.dbo.Contacts.[Contact ID] 

But remember -- if you are changing a lot of records in your database this could be constrained by the speed of your I/O

Answer 2

Below has a lot of nonsense work to do - so don't do it.

UPDATE a
SET a.[Account ID] = b.[Account ID]
FROM SFAX.dbo.Leads as a
left join SFAX.dbo.Accts as b on a.[Company   Account] = b.[Account Name]
WHERE a.[Account ID] IS NULL;

If there is no match in b, you will set [a.Account ID] to null. But you are only selecting those rows in a where the column is NULL. SO DON'T DO THAT - your left join is pointless extra work. Change it to an inner join. Will that help significantly? Can't tell without knowing the characteristics of your data.