AWK Multiple Field Separators and Variables

Question

I am trying to perform calculations in awk using fields whose numbers are passed in from the shell, as well as the last four fields

eg I call my shell script like this

./myProgram myFile.txt 1 2 3 4

Then within my shell script I want to use awk to refer to fields in a text file like this, specifically the last four fields. $(NF-3) - $(NF)

0000000022:trevor:736:1,2:3,4
0000000223:john:73:5,6:7,8
0000002224:eliza:54:9,8:7,6
0000022225:paul:22:5,4:3,2
0000222226:chris:0:1,2:3,4

So I can go through the fields, however when I do because there are two types of field separators it doesn't seem to work.

My shell script so far:

#! /usr/bin/env bash

file="$1"

awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "u1 =", $u1 }' $1
awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "v1 =", $v1 }' $1
awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "u2 =", $u2 }' $1
awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "v2 =", $v2 }' $1

echo "Argument #1 =" $2
echo "Argument #2 =" $3
echo "Argument #3 =" $4
echo "Argument #4 =" $5

This is the output I get from terminal:

u1 = 1
u1 = 5
u1 = 9
u1 = 5
u1 = 1
v1 = awk: illegal field $(), name "v1"
 input record number 1, file database.txt
 source line number 1
u2 = awk: illegal field $(), name "u2"
 input record number 1, file database.txt
 source line number 1
v2 = awk: illegal field $(), name "v2"
 input record number 1, file database.txt
 source line number 1
Argument #1 = 1
Argument #2 = 2
Argument #3 = 3
Argument #4 = 4

Answer 1

When you use $N in awk, it will retrieve field N . You can use this in combination with passing arguments to awk as you have done to access a field number defined in a shell variable. The main issue would appear to be that you are passing variables that haven't been set in your script.

In your example invocation of the script, you're not passing enough arguments for positional parameters $6 and above to be defined. This is what is causing your error messages that look like illegal field $() , because v1 is an empty string, so you're attempting to get a field with no number.

NF is a special variable in awk that contains the number of fields, so to access the last four fields, you can use $(NF-3) , $(NF-2) , $(NF-1) , and $NF .

There was a \\ before the awk command which wasn't doing anything useful, so I removed that as well.

There are a couple of other issues with your code that are worth mentioning too. Quote your shell variables! This prevents issues with word splitting on more complex variables. If your arguments are numbers with no spaces, this won't make any difference but it does no harm either and is a good practice to get into. You've defined file , so I've used that instead of $1 .

Combining those changes, we end up with something like this:

awk -F'[:,]' -v u1="$2" -v v1="$3" -v u2="$4" -v v2="$5" '{ print "u1 =", u1 }' "$file"

Answer 2

Just about one line:

awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \\ '{ print "u1 =", $u1 }' $1

Here $5,$6,$7 and $8 are the bash positional parameters not awk field position. Here you have 5 parameters to your script according to your command line:

./myProgram myFile.txt 1 2 3 4

$1 = myFile.txt
$2 = 1
$3 = 2
$4 = 3
$5 = 4
$6 = 
$7 =
$8 =

That's why awk alert your only on the call to $v1 as it's equivalent to $ and is not a field value.

If I understood properly your problem you wish to get the line where the 4 last paramters match thoose values:

awk -F'[:,]' '{ print "u1=",$(NF-3),"v1=",$(NF-2),"u2=",$(NF-1),"v2=",$NF }' "$1"

NF being the number of field, minus 3 give the 4 field before the end.