AWK Escape Characters Confusion

Question

So I've been doing some reading on AWK but I seem to be having a problem executing the following code:

curl http://website.com/users.csv | tac | tac | sed '101,400!d' | grep "female" |
awk -F',' '{ print "sudo useradd -Db /home/gender/female" " " $1 " " "-c" " " \"$5 " " $6\""}' |
bash

I'm trying to escape the double quotes immediately before $5 and after $6 so that I can either do a system call from AWK or just pipe to bash. For some reason my escape characters are not working. I read this GNU AWK summary on escape sequences to no avail.

How can I accomplish this?

The output should look like this:

sudo useradd -Db /home/gender/female dinessid1965 -c "Marina Propst"

And the input looks like this:

reatim,Shephoi8v,female,Ms.,Eija,Kankkunen,4721 Pearcy Avenue,Fort Wayne,IN,46804,,260-715-7242,2/22/84,Pisces,Dermatologist

Answer 1

Firstly, for the escaping: you make your life harder than necessary by quoting everything separately. Instead of

print "sudo useradd -Db /home/gender/female" " " $1 " " "-c" " " \"$5 " " $6\""

you could write

print "sudo useradd -Db /home/gender/female " $1 " -c " \"$5 " " $6\""

at which point it becomes a little more obvious that there is something off with the quoting: GNU Awk complains that the \\ is not the last character of the line.

You could write it like this, with the escaped quote \\" between quotes:

print "sudo useradd -Db /home/gender/female " $1 " -c \"" $5 " " $6 "\""

or, easier to read, specify a variable that contains the double quote (see the manual ):

awk -v dq='"' '{ print "sudo useradd -Db /home/gender/female " $1 " -c " dq $5 " " $6 dq }'

Secondly, your whole chain can be simplified: tac | tac tac | tac does nothing, and what grep and sed do can be done by awk:

curl http://website.com/users.csv |
awk -F, -v dq='"' '/female/ && NR >= 101 && NR <= 400 { \
 print "sudo useradd -Db /home/gender/female " dq $1 dq " -c " dq $5 " " $6 dq }' |
bash

Alternatively, to avoid some of the ugliness of adding spaces between variables using print , we could use printf (thanks to Ed Morton for the nudge):

curl http://website.com/users.csv |
awk -F, '/female/ && NR >= 101 && NR <= 400 { \
printf "sudo useradd -Db /home/gender/female \"%s\" -c \"%s %s\"\n", $1, $5, $6 }' |
bash

Notice how also the expansion of $1 is quoted in the last two commands to prevent side effects of characters special to the shell (or even malicious commands).

Answer 2

You're making this harder than necessary by involving awk at all. It's also dangerous because you're not quoting the contents of $1 , $5 , and $6 for the shell. If they contain any funny characters, you're in trouble.

You can do it entirely in bash, safely:

curl http://website.com/users.csv | sed '101,400!d' | grep "female" |
while IFS=, read -a words; do
    sudo useradd -Db /home/gender/female "${words[0]}" -c "${words[4]} ${words[5]}"
done

Although for more safety you might need to ensure that the username field doesn't contain anything fishy (eg start with ../ ). I don't know how much checking useradd does.

Answer 3

you're using too many quotes!

Here is a simple example you can change into your format

$ echo dinessid1965,two,three,four,Marina,Propst |
   awk -F, -v q='"' '{print "sudo ...", $1, "-c", q $5, $6 q}'  

sudo ... dinessid1965 -c "Marina Propst"