Ubuntu bash脚本重用输入变量

Question

I am trying to implement a bash script that reads input from a piped command. I want to process each line on the input and perform some processing on each line. Additionally, I want to reuse the input passed to the script. However, I have noticed that I am unable to reuse the input. Below is my script:

#!/bin/bash
while read line
do
    percentage=$(echo $line | awk 'printf $1')
    # perform some processing using the total i.e. percentage/total*100
done

This script is to be executed like this:

cat data.txt | grep "status" | ./myscript.sh

Sample data is:

1 STATUS
2 AUTHORISED
11 SENT

If the script is modified to something like this:

#!/bin/bash
input=$1
total=$(cut -f1 $1)
echo $total
while read $input
do
    percentage=$(echo $line | awk 'printf $1')
     # perform some processing using the total i.e. percentage/total*100
done

This script only ouputs the total and the while loop is not executed.

Multiple piped commands are required for this task, as this script will be used by many things so that it is kept generic and reusable. How can this be achived without storing the contents to a file?

Sample Input:

1 STATUS
2 AUTHORISED
11 SENT

Total would be 14. So each item ie 1/14*100,2/14*100 and 11/14*100.

Answer 1

This is easy:

awk 'NR==FNR{a+=$1; next;} {print $1*100/a;}' data.txt data.txt

For the grep operation part:

awk 'NR==FNR{if ($0 ~ pattern) a+=$1; next;} {if ($0 ~ pattern) print $1*100/a;}' pattern=STATUS data.txt data.txt

If the input is not in a file, we can cache it in awk itself.

awk '{a+=$1; b[NR]=$1;} END {for(i in b) print b[i]*100/a;}' data.txt

awk '$0 ~ pattern {a+=$1; b[NR]=$1;} END {for(i in b) print b[i]*100/a;}'  pattern=STATUS data.txt

some_command | awk '$0 ~ pattern {a+=$1; b[NR]=$1;} END {for(i in b) print b[i]*100/a;}'  pattern=STATUS

Answer 2

If you want to perform arithmetic based on the total, you need to know that total before performing your arithmetic. So you need two loops -- one to read the data and add up the total, and the other to process that data based on the total.

#!/usr/bin/env bash

# Prepare our variables
declare total=0
declare -a data=()
declare -a newdata=()

# Collect the array data and build our total...
while read value text; do
    data+=($value)
    ((total+=$value))
done

# Process the array, populating a new array with the results.
for i in "${!data[@]}"; do
    newdata[$i]=$(( 100 * ${data[$i]} / $total ))
done

# Show our results
declare -p data
declare -p newdata

Remember that bash only supports integer math, so it's important to use 100 * $val1 / $val2 rather than $val1 / $val2 * 100 . If you want greater precision, consider using external tools like bc or dc instead of doing your math in bash alone.

My results with the script above on the sample data in your question:

$ testme.sh < testme.txt
declare -a data=([0]="1" [1]="2" [2]="11")
declare -a newdata=([0]="7" [1]="14" [2]="78")

Answer 3

Your script won't work because your $1 arg is empty.

Use xargs to get the argument instead of using it as stdin. Also since the output could be multi line you can use -0 on xargs to use the null delimiter.

> cat data.txt | grep -i 'status' | xargs -0 ./myscript.sh

Then you need to actually calculate the total, I did that here using awk because I was being lazy, but there is probably an easier way.

You can then read input line by line using read.

I edited the script to work for me in a test environment, see below:

#!/bin/bash                                              
input="$1"                                               
echo "$input"                                            
total=$(echo "$input" | awk '{sum+=$1} END {print sum}') 
echo "$input" | while read line; do                      
      if [ ! -z "$line" ]; then                          
          num=$(echo $line | awk '{print $1}')           
          percentage=$((100 * num / total))              
          echo "$percentage"                             
      fi                                                 
done  

Answer 4

You are better off using a temporary file. But it's easy enough to store the data in memory.

(This script assumes you are totaling column one, and that column one in each line is a count of data points matching the status in column 2):

#!/bin/sh

data=$(cat)  # Read all of the input and store it
total=$(echo "$data" | awk '{s+=$1}END{print s}')
echo "$data" | while read count status; do
        printf "%s:\t%2.2f%%\n" \
        "$status" "$(echo "20k$count $total/100*p" | dc)"
done