mysql script bash loop

Question

I have a csv file contains list of numbers,

479856252|1329148400996|1329148405406|1329148412444|0|6042292534|6042292534|14016487697|0|6
479856262|1329148401487|1329148405287|1329148412444|0|6136992594|6136992594|14016487697|0|6
479856272|1329148405124|1329148405124|1329148412470|0|8883398128||4016487697|0||P|0|7
479856282|1329148114930|1329148117966|1329148412502|0|8006564132|8779918502|7657954163|2609

when I do awk -F'|' '{print $1;}' file.csv > file2.csv awk -F'|' '{print $1;}' file.csv > file2.csv

I get the following table IDs

12345
67891
32234
22345
12345

I have a for loop which works like this

for i in `cat file2.cvs`
do 
DELETE FROM TABLE1 WHERE ID = $i
done 

however this creates high load on my box, I'm hoping that I could split the operation (for each 100 records from file2.csv) perform the deletion and then sleep for 1 seconds the proceeds with remaining values till the end of file.

Your help is highly appreciated.

Thank you

Answer 1

Do it in one SQL command:

DELETE FROM TABLE1 WHERE ID in (`cat file2.cvs | tr '\n' ','` -1)

The -1 at the end is to cater for the trailing comma. If your input file has no line feed on the last line, you won't need the -1 .

Answer 2

Rather than sleeping for 1 second after every 100 deletes, you could keep your existing file structure and just sleep for 0.01 seconds after each delete to achieve basically the same thing.

Something like this:

for i in `cat file2.cvs`
do 
  mysql -e "DELETE FROM TABLE1 WHERE ID = ${i};"
  sleep .01
done 

Answer 3

Simply use an auxiliary variable to count the number of loops.

j=0
for i in `cat file2.cvs`
do

    if [ $j -eq 100 ]
    then
        sleep 1  
        j=0
    fi        

    j=$[ j+1 ]
done