Ubuntu (14 & 16) Bash errors with printf loops from input containing lowercase “n” characters

Question

I have some bash scripts I have been running on Ubuntu 14.04 and 16.04 for well over a year now. Some recent Ubuntu update has broken bash and I cannot figure out how to sort this out.

Example:

#!/bin/bash
INPUT=myinput.txt
OUTPUT=myoutput.txt
ACTION1="0;"

cat $INPUT | while read LINE
do
printf "\t\"~${LINE}\"\t\t$ACTION1\n" >> $OUTPUT
done

my script then loops through the file doing the printf statement as follows but this is the output produced

"~jetmo" 0;
"~spamme" 0;
"~baidu" 0;

example contents of myinput.txt

jetmon
spammen
baidu

Input lines containing a lowercase n character gets stripped out ?? Am I missing some major change in bash that occured, this started happening a week ago and is driving me insane now.

I have tried changing #!/bin/bash to #!/bin/sh with same results.

Also tried this method of looping through the input file and still get the same results.

#!/bin/bash
INPUT=myinput.txt
OUTPUT=myoutput.txt
ACTION1="0;"
while read LINE
do
printf "\t\"~${LINE}\"\t\t$ACTION1\n" >> $OUTPUT
done < $INPUT

Tried suggestion from comments using echo

echo -e -n "\t\"~${LINE}\"\t\t$ACTION1\n" >> $OUTPUT

and now I get this output

-e -n "~jetmo" 0;
-e -n "~spamme" 0;
-e -n "~baidu" 0;

Running hexdump -C myinput.txt gives this output

00000000  6a 65 74 6d 6f 6e 0a 73  70 61 6d 6d 65 6e 0a 62  |jetmon.spammen.b|
00000010  61 69 64 75 0a 0a                                 |aidu..|
00000016

Also changed all variable names to lowercase as suggested by Michael but still getting the same results.

#!/bin/bash
input=myinput.txt
output=myoutput.txt
action1="0;"

cat $input | while read line
do
printf "\t\"~${line}\"\t\t$action1\n" >> $output
done

THE SOLUTION The Mystery is Solved

thanks to everyone, but https://stackoverflow.com/users/96588/l0b0 nailed it on the head. I had an IFS=$'\\n' hiding earlier on in my script.

I now have this final, better and safer formatting of printf thanks to the recommendations of Nahuel and Charles and it is working 100% ... cannot thank you all enough.

#!/bin/bash
input=myinput.txt
output=myoutput.txt
action1="0;"
while IFS= read -r LINE
do
printf '\t"~%s"\t\t%s\n' "${LINE}" "$ACTION1" >> "$output"
done < $input1

Answer 1

This can happen if the internal field separator ( $IFS ) contains the letter n :

$ IFS=$' \t\nn' read line <<< foon
$ printf '%q\n' "$line"
foo

This is a fairly common mistake . Here is the correct value:

$ printf '%q\n' "$IFS"
$' \t\n'

Example:

$ IFS=$' \t\n' read line <<< foon
$ printf '%q\n' "$line"
foon

Answer 2

It's safer to use %s to insert a string with printf for example if it can contain %

printf "\t\"~%s\"\t\t%s\n" "${LINE}" "$ACTION1" >> $OUTPUT

EDIT following comments, with single quotes in first argument because there is no variable expansion

printf '\t"~%s"\t\t%s\n' "${LINE}" "$ACTION1" >> "$OUTPUT"

Answer 3

THE SOLUTION The Mystery is Solved

thanks to everyone, but https://stackoverflow.com/users/96588/l0b0 nailed it on the head. I had an IFS=$'\\n' hiding earlier on in my script.

I now have this final, better and safer formatting of printf thanks to the recommendations of Nahuel and Charles and it is working 100% ... cannot thank you all enough.

#!/bin/bash
input=myinput.txt
output=myoutput.txt
action1="0;"
while IFS= read -r LINE
do
printf '\t"~%s"\t\t%s\n' "${LINE}" "$ACTION1" >> "$output"
done < $input1