Using awk to read and create files in all subdirectories

Question

I am trying to parse all files named "README" in all subdirectories (and sub-subdirectories) under my specified directory, and create a new file containing the parsed output in the same directory where each "README" file was found.

#!/bin/bash

FILES=$(find myDirectory -type f -name 'README')
for f in $FILES
do
   #this is fine
   echo "parsing $f"

   #this is not fine
   awk -F, 'BEGIN {print "header"};
   {print $2;}
   END {print "footer";}' $f > outputfile

done

The output file is only being created in my working directory. What I would like this to do is to perhaps redirect the output files into the subdirectories where their corresponding README's were found. Is there a better way than this?

If it helps, README format:

something,something2,something3
nothing1,nothing2,nothing3

Answer 1

Given that you want the output file created in the directory where the README was found, the simplest way is to use the POSIX standard dirname command:

#!/bin/bash

FILES=$(find myDirectory -type f -name 'README')
for f in $FILES
do
    outputfile="$(dirname "$f")/outputfile"
    echo "parsing $f into $outputfile"

    awk -F, 'BEGIN {print "header"}
             {print $2}
             END {print "footer"}' "$f" > "$outputfile"

done

This code is not safe if there are spaces or newlines in the directories, but assuming you stick with the portable file name character set (letters, digits, dot, dash and underscore), there'll be no major problems. (It wasn't safe before I made any changes; it still isn't safe. It isn't safe because you used FILES=$(find …) and while you do that, it is pretty much guaranteed to remain unsafe for names with blanks, tabs, newlines in them. There are ways to fix that, but they involve more major surgery.)

If you want, you can study the Bash parameter expansion mechanisms to see how to do it without using dirname .