grep output different in bash script

Question

I am creating a bash script that will simply use grep to look through a bunch of logs for a certain string.

Something interesting happens though.

For the purpose of testing all of the log files the files are named test1.log, test2.log, test3.log, etc.

When using the grep command:

grep -oHnR TEST Logs/test*

The output contains all instances from all files in the folder as expected.

But when using a command but contained in the bash script below:

#!/bin/bash
#start

grep -oHnR $1 $2

#end

The output displays the instances from only 1 file.

When running the script I am using the following command:

bash test.bash TEST Logs/test*

Here is an example of the expected output (what occurs when simply using grep):

Logs/test2.log:8:TEST    
Logs/test2.log:20:TEST    
Logs/test2.log:41:TEST    
Logs/test.log:2:TEST    
Logs/test.log:18:TEST

and here is an example of the output received when using the bash script:

Logs/test2.log:8:TEST    
Logs/test2.log:20:TEST    
Logs/test2.log:41:TEST

Can someone explain to me why this happens?

Answer 1

When you call the line

bash test.bash TEST Logs/test*

this will be translated by the shell to

bash test.bash TEST Logs/test1.log Logs/test2.log Logs/test3.log Logs/test4.log

(if you have four log files).

The command line parameters TEST , Logs/test1.log , Logs/test2.log , etc. will be given the names $1 , $2 , $3 , etc.; $1 will be TEST , $2 will be Logs/test1.log .

You just ignore the remaining parameters and use just one log file when you use $2 only.

A correct version would be this:

#!/bin/bash
#start

grep -oHnR "$@"

#end

This will pass all the parameters properly and also take care of nastinesses like spaces in file names (your version would have had trouble with these).

Answer 2

To understand what's happening, you can use a simpler script:

#!/bin/bash
echo $1
echo $2

That outputs the first two arguments, as you asked for.

You want to use the first argument, and then use all the rest as input files. So use shift like this:

#!/bin/bash
search=$1
shift

echo "$1"
echo "$@"

Notice also the use of double quotes.

In your case, because you want the search string and the filenames to be passed to grep in the same order, you don't even need to shift :

#!/bin/bash

grep -oHnR -e "$@"

(I added the -e in case the search string begins with - )

Answer 3

The unquoted * is being affected by globbing when you are calling the script.

Using set -x to output what is running from the script makes this more clear.

$ ./greptest.sh TEST test*
++ grep -oHnR TEST test1.log
$ ./greptest.sh TEST "test*"
++ grep -oHnR TEST test1.log test2.log test3.log

In the first case, bash is expanding the * into the list of file names versus the second case it is being passed to grep. In the first case you actually have >2 args (as each filename expanded would become an arg) - adding echo $# to the script shows this too:

$ ./greptest.sh TEST test*
++ grep -oHnR TEST test1.log
++ echo 4
4
$ ./greptest.sh TEST "test*"
++ grep -oHnR TEST test1.log test2.log test3.log
++ echo 2
2

Answer 4

You probably want to escape the wildcard on your bash invocation:

bash test.bash TEST Logs/test\*

That way it'll get passed through to grep as an * , otherwise the shell will have expanded it to every file in the Logs dir whose name starts with test .

Alternatively, change your script to allow more than one file on the command line:

#!/bin/bash
hold=$1
shift
grep -oHnR $hold $@