循环遍历目录中的每个文件 - bash

Question

I'm trying to perform certain operation on each file in a directory but there is a problem with order it's going through. It should do one file at the time. The long line (unzipping, grepping, zipping) works fine on a single file without a script, so there is a problem with a loop. Any ideas?

Script should grep through through each zipped file and look for word1 or word2. If at least one of them exist then:

1. unzip file
2. grep word1 and word2 and save it to file_done
3. remove unzipped file
4. zip file_done to /donefiles/ with original name
5. remove file_done from original directory

    #!/bin/bash
    for file in *.gz; do
    counter=$(zgrep -c 'word1\|word2' $file)
    if [[  $counter -gt 0 ]]; then
    echo $counter
    for file in *.gz; do
    filenoext=${file::-3}
    filedone=${filenoext}_done
    echo $file
    echo $filenoext
    echo $filedone
    gunzip  $file | grep 'word1\|word2'  $filenoext > $filedone | rm -f $filenoext |  gzip -f  -c  $filedone > /donefiles/$file | rm -f $filedone
    done
    else
    echo "nothing to do here"
    fi
    done

Answer 1

The code snipped you've provided has a few problems, eg unneeded nested for cycle and erroneous pipeline (the whole line gunzip $file | grep 'word1\\|word2' $filenoext > $filedone | rm -f $filenoext | gzip... ).

Note also your code will work correctly only if *.gz files don't have spaces (or special characters) in names. Also zgrep -c 'word1\\|word2' will also match strings like line_starts_withword1_orword2_ .

Here is the working version of the script:

#!/bin/bash
for file in *.gz; do
        counter=$(zgrep -c -E 'word1|word2' $file) # now counter is the number of word1/word2 occurences in $file
        if [[ $counter -gt 0 ]]; then
                name=$(basename $file .gz)
                zcat $file | grep -E 'word1|word2' > ${name}_done
                gzip -f -c ${name}_done > /donefiles/$file
                rm -f ${name}_done
        else
                echo 'nothing to do here'
        fi
done

What we can improve here is:

• since we unzipping the file anyway to check for word1|word2 presence, we may do this to temp file and avoid double-unzipping
• we don't need to count how many word1 or word2 is inside the file, we may just check for their presence
• ${name}_done can be a temp file cleaned up automatically
• we can use while cycle to handle file names with spaces

#!/bin/bash
tmp=`mktemp /tmp/gzip_demo.XXXXXX` # create temp file for us
trap "rm -f \"$tmp\"" EXIT INT TERM QUIT HUP # clean $tmp upon exit or termination
find . -maxdepth 1 -mindepth 1 -type f -name '*.gz' | while read f; do
        # quotes around $f are now required in case of spaces in it
        s=$(basename "$f") # short name w/o dir
        gunzip -f -c "$f" | grep -P '\b(word1|word2)\b' > "$tmp"
        [ -s "$tmp" ] && gzip -f -c "$tmp" > "/donefiles/$s" # create archive if anything is found
done

Answer 2

It looks like you have an inner loop inside the outer one :

#!/bin/bash
for file in *.gz; do
    counter=$(zgrep -c 'word1\|word2' $file)
    if [[  $counter -gt 0 ]]; then
        echo $counter
        for file in *.gz; do #<<< HERE
            filenoext=${file::-3}
            filedone=${filenoext}_done
            echo $file
            echo $filenoext
            echo $filedone
            gunzip  $file | grep 'word1\|word2'  $filenoext > $filedone | rm -f $filenoext |  gzip -f  -c  $filedone > /donefiles/$file | rm -f $filedone
        done
    else
        echo "nothing to do here"
    fi
done

The inner loop goes through all the files in the directory if one of them contains file1 or file2. You probably want this :

#!/bin/bash
for file in *.gz; do
    counter=$(zgrep -c 'word1\|word2' $file)
    if [[  $counter -gt 0 ]]; then
        echo $counter
        filenoext=${file::-3}
        filedone=${filenoext}_done
        echo $file
        echo $filenoext
        echo $filedone
        gunzip  $file | grep 'word1\|word2'  $filenoext > $filedone | rm -f $filenoext |  gzip -f  -c  $filedone > /donefiles/$file | rm -f $filedone
    else
        echo "nothing to do here"
    fi
done