Tar backup strange behaviour in bash script

Question

I have bash script used to take backup of / directories and i am using pipe viewer(pv) to see the progress bar .When i run the command manually it works fine but when I run same command in bash script progress bar shows it execeeds 100% and ends up at 108% although ETA shows 00:00 when it reaches 100% but it still goes on?? Here is my script

DIR_1="/var /root /sbin /bin /etc /lib /www /usr /mnt";

CAL_1=$(du -skc $DIR_1 | awk '{print $1}' | tail -n1);

    if [ "$1" = "fullbackup" ]                                                                            
    then                                                                                                  
            echo "`date +%F\ %H:%M` backup started..">>$FLOG;                                             
            /bin/nice -n 19 tar cf - `echo $DIR_1` | pv -s ${CAL_1}k >$DESTINATION/$FILENAME -f 2>$TARLOG;
            if [ $? -eq 0 ]                                                                               
            then                                                                                          
                    echo "`date +%F\ %H:%M` tar archive successfull" >>$FLOG ;                            
                    NEW=$(cd $DESTINATION && ls -t full* | head | sed '1!d');                             

            elif [ $? -ne 0 ]                                                                             
            then                                                                                          
                    echo "`date +%F\ %H:%M` tar archive failed">>$FLOG;                                   
                    rm $DESTINATION/$FILENAME;                                                            
                    exit;                                                                                 
            fi                                                              

Answer 1

Here's what's going on. You are telling pv that the size of the tar file is of sum of the files under $DIR_1 and sometimes it is not.

In particular, when you run the program and you get 108% rather than 100%, that the 8% difference is the the overhead of the tar file. You can easily verify this by comparing the value you get back with $CAL_1 with what you get back with a ls -l $DESTINATION/$FILENAME .

There can also be some slight discrepancies in the way du -k decides what K means versus how pv measures it. To eliminate that possibility remove the 'k' from both pv and du .

I think running this interactively is a red herring. I'm guessing when you ran interactively you ran on a small filesystem where the tar overhead was close to 0, while when you run in a script it was done on a large filesystem. (Small filesystems is what one would do interactively because who wants to wait hours to get results?).

To verify to eliminate the interactive versus script situation, make sure you run the two on the same filesystem.

If this doesn't do it, I'd suggest adding to your question specific output mentioned above such as what the value of $CAL_1 is and what an ls of the final tar file is.