Is it possible to set variable in pipeline?

Question

I have a big txt file which I want to edit in pipeline. But on same place in pipeline I want to set number of lines in variable $nol. I just want to see sintax how could I set variable in pipeline like:

  cat ${!#} | tr ' ' '\n'| grep . ; $nol=wc -l | sort | uniq -c ...

That after second pipe is very wrong, but how can I do it in bash?

One of solutions is:

nol=$(cat ${!#} | tr ' ' '\n'| grep . | wc -l)
pipeline all from the start again

but I don't want to do script the same thing twice, bec I have more pipes then here.

I musn't use awk or sed...

Answer 1

You can set a variable in a particular link of a pipeline, but that's not very useful since only that particular link will be affected by it.

I recommend simply using a temporary file.

set -e
trap 'rm -f "$tmpf"' EXIT
tmpf=`mktemp`
cat ${!#} | tr ' ' '\n'| grep . | sort > "$tmpf"
nol="$(wc "$tmpf")"
< "$tmpf" uniq -c ...

You can avoid the temporary file with tee and a named pipe, but it probably won't perform much better (it may even perform worse).

Answer 2

Looks to me like you're asking how to avoid stepping through your file twice, just to get both word and line count.

Bash lets you read variables, and wc can produce all the numbers you need at once.

NAME
     wc -- word, line, character, and byte count

So to start...

read words line chars < <( wc < ${!#} )

This populates the three variables based on input generated from process substitution .

But your question includes another partial command line which I think you intend as:

nol=$( sort -u ${!#} | wc -l )

This is markedly different from the word count of your first command line, so you can't use a single wc instance to generate both. Instead, one option might be to put your functionality into a script that does both functions at once:

read words uniques < <(
awk '
      {
        words += NF
        for (i=1; i<=NF; i++) { unique[$i] }
      }
      END {
        print words,length(unique)
      }
    ' ${!#}
)

Answer 3

You can use a tee and then write it to a file which you use later:

tempfile="xyz"
tr ' ' '\n' < "${!#}" | grep '.' | tee > "$tempfile" | sort | uniq -c ...
nol=$(wc -l "$tempfile")

Or you can use it the other way around:

nol=$(tr ' ' '\n' < "${!#}" | grep '.' \
  | tee >(sort | uniq -c ... > /dev/tty) | wc -l

Answer 4

UPDATE:

Took a minute but I got it...

cat ${!#} | tr ' ' '\n'| tee >(nol=$(wc -l)) | sort | uniq -c ...

PREVIOUS:

The only way I can think to do this is storing in variables and calling back. You would not execute the command more than one time. You would just store the output in variables along the way.

aCommand=($(cat ${!#} | tr ' ' '\n'));sLineCount=$(echo ${#aCommand[@]});echo ${aCommand[@]} | sort | uniq -c ...

• aCommand will store the results of the first set of commands in an array
• sLineCount will count the elements (lines) in the array
• ;... echo the array elements and continue the commands from there.