将postgres中的简单更新语句加速一百万行

Question

I have a very simple sql update statement in postgres.

UPDATE p2sa.observation SET file_path = replace(file_path, 'path/sps', 'newpath/p2s')

The observation table has 1513128 rows. The query so far has been running for around 18 hours with no end in sight.

The file_path column is not indexed so I guess it is doing a top to bottom scan but it seems a bit excessive the time. Probably replace is also a slow operation.

Is there some alternative or better approach for doing this one off kind of update which affects all rows. It is essentially updating an old file path to a new location. It only needs to be updated once or maybe again in the future.

Thanks.

Answer 1

In SQL you could do a while loop to update in batches.

Try this to see how it performs.

Declare @counter int 
Declare @RowsEffected int 
Declare @RowsCnt int 
Declare @CodeId int 
Declare @Err int
DECLARE @MaxNumber int = (select COUNT(*) from p2sa.observation)
SELECT @COUNTER = 1
SELECT @RowsEffected = 0


WHILE ( @RowsEffected < @MaxNumber)
BEGIN 
SET ROWCOUNT 10000


UPDATE p2sa.observation 
SET file_path = replace(file_path, 'path/sps', 'newpath/p2s')
where file_path != 'newpath/p2s'

SELECT @RowsCnt = @@ROWCOUNT ,@Err = @@error 
IF @Err <> 0 
BEGIN 
Print 'Problem Updating the records' 
BREAK
END 
ELSE 

SELECT @RowsEffected = @RowsEffected + @RowsCnt 
PRINT 'The total number of rows effected :'+convert(varchar,@RowsEffected) 

            /*delaying the Loop for 10 secs , so that Update is completed*/    

     WAITFOR DELAY '00:00:10'         
 END
 SET ROWCOUNT 0