Search for multiple patterns in multiple files

Question

This is related to Function to search of multiple patterns using grep

I want to search multiple files with multiple patterns using command such as follows:

myscript *.txt pattern1 pattern2 pattern3

I tried implementing the codes in the previous question but they do not work with wildcards. For example, following does not work:

#!/bin/bash

ARGS=$# 
if [ $ARGS -lt 2 ]
then 
    echo "You entered only $ARGS arguments- at least 2 are needed."
    exit 
fi

search() {
    if [ $# -gt 0 ]
    then
        local pat=$1
        shift
        grep -i "$pat" | search "$@"
    else 
         cat
    fi
}

for VAR in $1
do 
    file=$VAR
    shift
    cat "$file" | search "$@"
done

How can I create a script which can search for multiple files (taking it from first argument) to search multiple patterns (from rest of arguments)?

Answer 1

Did you try to use find and sed ?

find . -name *.txt -exec sed -n -e '/pattern1/p' -e '/pattern2/' '{}' ';'

The -n option will make sure sed does not print all the file, and the p command prints the matching lines. Finaly, find will get all the files you need.

EDIT:

If you want to put that in a script to generate the sed command, you can use this trick .

EDIT 2:

As @shellter said, it is usually better to use options, and as your script is written, *.txt will be expanded by bash . To avoid that, you'll need to quote the first argument.

As usual, there is several solutions to your problem:

---

Solution 1 (Using bash built-in):

#! /usr/bin/env bash

set -o nounset # Throw error if variable not set
set -o errexit # Exit if error is thrown

work_dir=$PWD # directory to search from

# Reading the command line
files_pattern=${1:-}; # Save first argument as files pattern.
shift 1;              # Move $1 to next argument (and propagate such as $n gets $n+1)
echo "==> Files to search follow pattern: ${files_pattern}"

_len=$#; #save the number of arguments.
for (( i=0; i<$_len; i=$i+1 )); # Go through the search patterns.
do
    search_patterns[$i]=$1; # store the next search pattern
    shift 1; # move $1 to next patern.
    echo "==> New search pattern #$i: ${search_patterns[$i]}"
done

while read -r file; # Go through all the matching files
do
    echo "==> In file: ${file}"
    while read -r line; # Go though all the lines in the file
    do
        for regex in "${search_patterns[@]}"; # iterate trough patterns
        do
            [[ "${line}" =~ $regex ]] && echo "${line}";
        done
    done < ${file}
done < <(find $work_dir -iname $files_pattern -print) # find all the files matching file_pattern

---

Solution 2 (using grep):

#! /usr/bin/env bash

set -o nounset # Throw error if variable not set
set -o errexit # Exit if error is thrown

work_dir=$PWD # directory to search from

# Reading the command line
files_pattern=${1:-}; # Save first argument as files pattern.
shift 1;              # Move $1 to next argument (and propagate such as $n gets $n+1)
echo "==> Files to search follow pattern: ${files_pattern}"

while [ $# -gt 0 ]; # Go through the search patterns.
do
    search_patterns+="$1"; # store the next search pattern
    shift 1; # move $1 to next patern.
    [ $# -gt 0 ] && search_patterns+="|" #Add or option
done
echo "==> Search patterns: ${search_patterns}"

cd ${work_dir} && egrep -iR '('"${search_patterns}"')' && cd -;

---

Solution 3 (Using sed):

#! /usr/bin/env bash

set -o nounset # Throw error if variable not set
set -o errexit # Exit if error is thrown

work_dir=$PWD # directory to search from

# Reading the command line
files_pattern=${1:-}; # Save first argument as files pattern.
shift 1;              # Move $1 to next argument (and propagate such as $n gets $n+1)
echo "==> Files to search follow pattern: ${files_pattern}"

while [ $# -gt 0 ]; # Go through the search patterns.
do
    search_patterns+="/$1/p;"; # store the next search pattern
    shift 1; # move $1 to next patern.
    [ $# -gt 0 ] && search_patterns+=" " #Add or option
done
echo "==> Search patterns: ${search_patterns}"

# Will print file names, and then matching lines
find "$work_dir" -iname "$files_pattern" -print -exec sed -n "${search_patterns}" '{}' ';'

---

I am sure there is plenty other ways to tweak or solve this problem, but this should get you started.

Good Luck!