While loop (in a background process) reading from file descriptor does not exit

Question

I'll explain my problem statement first. I have a command1 which generates data on both stderr and stdout, the stdout is piped to command2, while the stderr should go to a background process which continuously calls an API.

Dummy example of the same:

#!/bin/bash

set -e;

metric_background_process () {
    # This while loop waits till there is a line to read
    while read -u 3 -r line; do
        echo $line;
    done;
    
    # This never prints !!!
    echo "Done"
}

tmp_dir=$(mktemp -d)
mkfifo "$tmp_dir/f1"

# Allocate a file descriptor to the named pipe
# to allow read and write from it.
exec 3<> "$tmp_dir/f1"

metric_background_process <&3 &
pid=$!


# Main commands (stdout is being piped to stderr as a dummy case)
cat ./assets/random_file 1>&3

exec 3<&-

wait $pid

The input file contains: 1 through 9, with 9 being the last line.

Observed Output:

1
2
3
4
5
6
7
8

The shell waits after printing 8 , 9 is not printed, and the program does not stop by itself.

No matter what I try, the while loop does not exit. Maybe I'm missing something simple, any help or further questions for clarification would be highly appreciated.

Answer 1

The parent shell is opening the fifo for read/write ( <> ). The subshell ( & ) inherits the FD so it's also opening the fifo for read/write. When the parent closes the FD the subshell is still opening the fifo for writing so the write side is never closed and so the read side ( read -u 3 ) cannot get EOF .

To make it a bit simpler —

The script:

$ cat foo.sh 
metric_background_process () {
    local fifo=$1

    while read line; do
        echo $line
    done < $fifo
    
    echo "Done"
}

tmp_dir=$(mktemp -d)
fifo="$tmp_dir/f1"
mkfifo "$tmp_dir/f1"

metric_background_process $fifo &
pid=$!

exec 3> $fifo

for i in {1..5}; do
    echo i=$i >&3
done

exec 3>&-

wait $pid

Result:

$ bash foo.sh
i=1
i=2
i=3
i=4
i=5
Done