Bash：使用bash脚本的头部和尾部行为

Question

Suppose I have following script:-

test.sh

#!/bin/bash
command1  #prints 5 lines
command2  #prints 3 lines

I run the script with test.sh|head -n5

What will happen in this case? Will it run both the commands? or will it stop after command1? What if I call it with -n1 ?

Background: I might be asking a very basic question, but I actually noticed something interesting. My script(different one) was processing 7,000 files and each file produces 1 line of output. It takes 7 minutes to run the script completely but doing head -n1 gave me prompt immediately like the script has terminated after processing first file only

Edit: Following is my script

for i in $(ls filepath);do
     echo "$i" # issue here
    python mySript "$i" > "/home/user/output/""$i"".out"
  fi
done

Removing echo above enables the script to run full 7 minute with head -n1 , but with echo it just prints first line then exit.

Answer 1

This is a fairly interesting issue! Thanks for posting it!

I assumed that this happens as head exits after processing the first few lines, so SIGPIPE signal is sent to the bash running the script when it tries to echo $x next time. I used RedX's script to prove this theory:

#!/usr/bin/bash
rm x.log
for((x=0;x<5;++x)); do
    echo $x
    echo $x>>x.log
done

This works, as You described! Using t.sh|head -n 2 it writes only 2 lines to the screen and to x.log. But trapping SIGPIPE this behavior changes...

#!/usr/bin/bash
trap "echo SIGPIPE>&2" PIPE
rm x.log
for((x=0;x<5;++x)); do
    echo $x
    echo $x>>x.log
done

Output:

$ ./t.sh |head -n 2
0
1
./t.sh: line 5: echo: write error: Broken pipe
SIGPIPE
./t.sh: line 5: echo: write error: Broken pipe
SIGPIPE
./t.sh: line 5: echo: write error: Broken pipe
SIGPIPE

The write error occurs as stdout is already closed as the other end of the pipe is closed. And any attempt to write to the closed pipe causes a SIGPIPE signal, which terminates the program by default (see man 7 signal ). The x.log now contains 5 lines.

This also explains why /bin/echo solved the problem. See the following script:

rm x.log
for((x=0;x<5;++x)); do
    /bin/echo $x
    echo "Ret: $?">&2
    echo $x>>x.log
done

Output:

$ ./t.sh |head -n 2
0
Ret: 0
1
Ret: 0
Ret: 141
Ret: 141
Ret: 141

Decimal 141 = hex 8D. Hex 80 means a signal was received, hex 0D is for SIGPIPE. So when /bin/echo tried to write to stdout it got a SIGPIPE and it was terminated (as default behavior) instead of the bash running the script.

Answer 2

Nice finding. According to my tests it's exactly like you said. For example I have this script that just eats cpu, to let us spot it in top :

for i in `seq 10`
  do echo $i
  x=`seq 10000000`
done

Piping the script with head -n1 we see the command returning after the first line. This is the head behavior: it completed its work, so it can stop and return the control to you.

The input script should continue running but look what happens: when the head returns, its pid doesn't exist anymore. So when linux tries to send the output of the script to the head process, it does not find the process, so the script crashes and stops.

Let's try it with a python script:

for i in xrange(10):
    print i
    range(10000000)

When running it and piping to head you have this:

$ python -u test.py | head -n1
0
Traceback (most recent call last):
  File "test.py", line 2, in <module>
    print i
IOError: [Errno 32] Broken pipe

The -u option tells python to automatically flush the stdin and stdout, as bash would do. So you see that the program actually stops with an error.

Answer 3

This is more of a comment then an answer but it is too big for a comment.

I tried following script:

#!/usr/bin/env bash

rm -f "test_head.log"
echo "1 line"
echo "1 line" >> "test_head.log"
echo "2 line"
echo "2 line" >> "test_head.log"
echo "3 line"
echo "3 line" >> "test_head.log"
echo "4 line"
echo "4 line" >> "test_head.log"
echo "5 line"
echo "5 line" >> "test_head.log"
echo "6 line"
echo "6 line" >> "test_head.log"
echo "7 line"
echo "7 line" >> "test_head.log"
echo "8 line"   
echo "8 line" >> "test_head.log"

Then i ran the script with:

./test_head.sh | head -n1

The cat output is (to my surprise):

1 line

I have no idea what is going on.

After reading @ymonad comment i tried it out and replace echo with /bin/echo and that solved the problem. I hope he can explain more about this behaviour.