Unable to grep output of a command in bash script

Question

The below piece of script is not behaving as expected

if docker pull docker.pkg.github.com/private-repo/centos7 | grep -q 'Error response from daemon: unauthorized'; then
  echo "matched"
else 
  echo "unmatched"
fi

output

Error response from daemon: unauthorized
unmatched

expected output

matched

I have followed this post

What i have tried: i replaced "docker.pkg.github.com/private-repo/centos7" with echo "Error response from daemon: unauthorized" and it gives expected o/p as matched.

so, what i understand here is the o/p from command "docker pull docker.pkg.github.com/private-repo/centos7" is not captured by "grep" but i don't understand why?

I've also tried this but same result:

docker pull docker.pkg.github.com/private-repo/centos7 | grep 'Error response from daemon: unauthorized' &> /dev/null
if [ $? == 0 ]; then
   echo "matched"
else
   echo "unmatched"
fi

Working solution suggested by @Gordon Davisson

docker pull docker.pkg.github.com/private-repo/centos7 2>&1 | grep 'Error response from daemon: unauthorized' &> /dev/null
if [ $? == 0 ]; then
   echo "matched"
else
   echo "unmatched"
fi

output: matched

Answer 1

It's just as @Gordon Davisson said, and please give him the answer credit if he chooses to claim. I'm making the answer more visible.

This is an oversimplification, but it will get the point across. All “outputs” are sent to the terminal through stdout and stderr.

When you use the basic pipe-syntax (|), the only thing actually being processed by the pipe is the stdout. The stderr will still be printed to the terminal. In your case this is undesirable behavior.

The fix is to force the stderr into the stdout BEFORE the pipe, the syntax for this is 2&>1 (or in Bash |&). This works around the pipe's limitation of only being able to process stdout, and it also prevents the stderr leak into the terminal.

if docker pull… 2&>1 | grep -q…
<SNIPPED>

OR IN BASH

if docker pull… |& grep -q…
<SNIPPED>

The reason your 2nd-attempted solution didn't work was because pipes and redirections are processed in-order from left-to-right.

if docker pull… | grep… &> /dev/null
#              ^ LEAK HAPPENS HERE, FIX COMES TOO LATE
<SNIPPED>

Meaning that the stderr leak into the terminal already occurred BEFORE you redirected grep's output. And that the error wasn't occurring from grep.

Answer 2

The issue is that the output of the docker pull command is not being passed to the grep command. Instead, the output is being printed to the console. To fix this, you can use command substitution to assign the output of the docker pull command to a variable, then pass that variable to grep like this:

    output=$(docker pull docker.pkg.github.com/private-repo/centos7)
if echo "$output" | grep -q 'Error response from daemon: unauthorized'; then
  echo "matched"
else 
  echo "unmatched"
fi

This will assign the output of the docker pull command to the variable output, then pass that variable to grep for comparison. The echo command is used to pass the variable to grep.

Answer 3

You might have some luck with just searching for Error instead of the whole string and see if you got something wrong with the way you typed out the string.