Columns and Arguments AWK Parameters in Shell Scripting

Question

I want to get positional parameters as arguments for my .sh file and I also want to get fields from a text file for awk. I've figured out that I need to use $1-$9 for both and its okay to use the same number $() in awk as a positioning parameter, it still works.

eg I call my shell script like this

./myProgram myFile.txt 1 2 3 4

Then within my shell script I want to use awk to refer to fields in a text file like this, specifically 1,2:3,4 the last four fields.

0000000022:trevor:736:1,2:3,4
0000000223:john:73:5,6:7,8
0000002224:eliza:54:9,8:7,6
0000022225:paul:22:5,4:3,2
0000222226:chris:0:1,2:3,4

So I can go through the fields, however when I do because there are two types of field separators it doesn't seem to work.

My shell script so far:

#! /usr/bin/env bash

file="$1"

awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "u1 =", $u1 }' $1
awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "v1 =", $v1 }' $1
awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "u2 =", $u2 }' $1
awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "v2 =", $v2 }' $1

echo "Argument #1 =" $2
echo "Argument #2 =" $3
echo "Argument #3 =" $4
echo "Argument #4 =" $5

This is the output I get from terminal:

u1 = 1
u1 = 5
u1 = 9
u1 = 5
u1 = 1
v1 = awk: illegal field $(), name "v1"
 input record number 1, file database.txt
 source line number 1
u2 = awk: illegal field $(), name "u2"
 input record number 1, file database.txt
 source line number 1
v2 = awk: illegal field $(), name "v2"
 input record number 1, file database.txt
 source line number 1
Argument #1 = 1
Argument #2 = 2
Argument #3 = 3
Argument #4 = 4

I'm so close yet so far, I'm not sure why I'm not able to go further across the fields using my awk script?

Answer 1

Update : It seems that the OP's problem stemmed from confusion over the distinction between shell parameters ( $1 , $2 , ...) with input-field variables in Awk - which can look the same, but are entirely unrelated.
Specifically, the incorrect assumption was made that if n parameters were passed to the shell script, Awk 's input-field numbering would start with n+1 .

---

The following snippet - originally written before the OP added more code to the question - demonstrates the interplay of shell parameters and Awk variables , followed by a detailed explanation.

Specifically, it defines Awk variables fi1 and fi2 based on the values of shell parameters $2 and $3 respectively, which contain 1-based field indices relative to the lines in file $file .

Then, inside the Awk program, the field indices stored in fi1 and fi2 are used first as is (no $ prefix) to print their own value, and then to reference the corresponding input-line fields by prepending $ (variable references in Awk are NOT $ -prefixed - the $ is only used to refer to fields ).

#!/usr/bin/env bash

file="$1"

awk -F'[:,]' -v fi1=$2 -v fi2=$3 \
  '{ print "Field #" fi1 " + field #" fi2 " =", $fi1 + $fi2 }' "$file"

• The shell and awk are separate worlds, and do not see each other's variables.
  • You can see environment variables in awk , by accessing the ENVIRON associative array, but you cannot see shell variables.
  • You can implicitly "bake" shell-variable values into an awk program by passing a double-quoted string with shell-variable references - which are expanded before awk sees the program - but that gets confusing quickly and should be avoided.
  • Conversely, the only way to pass values back from awk to the shell is to have awk print to stdout and use a shell command substitution to capture the result in a shell variable.
• You can pass the values of shell parameters and variables to awk variables using instances of the -v option, as demonstrated above.
  • Here we know that the values are numbers, so we don't strictly need to double-quote the var. references, but it in general it's advisable.
• Inside awk , its variables are referenced without the $ prefix, and $ is only used to refer to input fields :
  • $fi1 , perhaps somewhat confusingly, therefore means: get the input field (prefix $ ) whose index is stored in awk variable fi1 (by contrast, using fi1 as is, without a prefix, would return the index ).
• Field indices in awk are always 1 -based , relative to each line of input.
  • For instance, inside awk , $1 refers to the first field on the current input line , and even though it looks the same as the 1st script / function parameter in the shell , they have absolutely nothing to do with each other.
  • Additionally, $0 contains the entire current input line, and NF contains the number (count) of input fields.