Why does my awk command print an extra line?

Question

I'm finding it difficult to debug my shell scripts. I have the following file test.csv

The Gardens,Gard (AUS),AEST,NSW,,Sandown Park,SPrk (AUS),AEST,VIC,,Grade 5,GR5,
Wentworth Park,WPrk (AUS) ,AEST,NSW,,The Meadows,Mead (AUS),AEST,VIC,,Juvenille,JUV,
Angle Park,AnPk (AUS),ACST,SA,,Warragul,Wgul (AUS),AEST,VIC,,,,

WPrk (AUS) has an extra trailing white space which I wish to trim and then print. Running

awk -F\, 'gsub(/[ \t]+$/, "", $2); {print $2 ":"}' test.csv

Produces

Gard (AUS):
Wentworth Park WPrk (AUS) AEST NSW  The Meadows Mead (AUS) AEST VIC  Juvenille JUV
WPrk (AUS):
AnPk (AUS):

Which is what I want except for the second line I can't figure out why that appears

Answer 1

awk -F\, 'gsub(/[ \t]+$/, "", $2); {print $2 ":"}' test.csv

Since you have gsub outside any action block ( {...} ), it is evaluated every line as a condition . It returns the number of substitutions it made, so it will return 0 on most of the inputs, but 1 on the lines you want to change. Since there is no action accompanying it, the default action is executed, printing the line; that is why you are getting the full line printed.

The fix, as others have said, is putting the gsub call inside the action block.

(Also, why \\, ? Comma is not a special character in the shell.)

Answer 2

How about changing the command to be:

    awk -F, '{gsub(/[ \t]+$/, "", $2); print $2 ":"}' test.csv

That's moving '{' to the front.

Edited As others said, you don't need to escape comma, -F, is enough. Thanks to other answerers :).

Answer 3

You wrote:

awk -F\, 'gsub(/[ \t]+$/, "", $2); {print $2 ":"}' test.csv

This is broken.

The format for an awk script is a bunch of constructs that look like:

condition {
     command; 
     ... 
    }

The idea here is that each line in the input data is evaluated against condition . If it matches, then the commands in the corresponding curly brackets are executed. So what you probably want is more along the lines of this:

awk -F, '{gsub(/[ \t]+$/, "", $2); print $2 ":";}' test.csv

Note that you don't need to escape the field separator unless your shell will treat it badly (ie if it was a vertical bar, | , instead of a comma). By excluding the condition inside the script, you execute the curly-braced commands on EVERY line. So this will trim $2 whether it needs to be trimmed or not, then print the result per your example in your question.