how to iterate over files using find in bash/ksh shell

Question

I am using find in a loop to search recursively for files of a specific extension, and then do something with that loop.

cd $DSJobs
jobs=$(find $DSJobs -name "*.dsx")
for j in jobs; do
    echo "$j"
done

assuming $DSJobs is a relevent folder, the output of $j is "Jobs" one time. doesn't even repeat. I want to list all *.dsx files in a folder recursively through subfolders as well.

How do Make this work? Thanks

Answer 1

The idiomatic way to do this is:

cd "$DSJobs"
find . -name "*.dsx" -print0 | while IFS= read -r -d "" job; do
    echo "$job"
done

The complication derives from the fact that space and newline are perfectly valid filename characters, so you get find to output the filenames separated by the null character (which is not allowed to appear in a filename). Then you tell read to use the null character (with -d "" ) as the delimiter while reading the names.

IFS= read -r var is the way to get bash to read the characters verbatim, without dropping any leading/trailing whitespace or any backslashes.

There are further complications regarding the use of the pipe, which may or may not matter to you depending on what you do inside the loop.

Note: take care to quote your variables, unless you know exactly when to leave the quotes off. Very detailed discussion here .

---

Having said that, bash can do this without find :

shopt -s globstar
cd "$DSJobs"
for job in **/*.dsx; do
    echo "$job"
done

This approach removes all the complications of find | while read find | while read .

---

Incorporating @Gordon's comment:

shopt -s globstar nullglob
for job in "$DSJobs"/**/*.dsx; do
    do_stuff_with "$job"
done

The "nullglob" setting is useful when no files match the pattern. Without it, the for loop will have a single iteration where job will have the value job='/path/to/DSJobs/**/*.dsx' (or whatever the contents of the variable) -- including the literal asterisks.

Answer 2

Since all you want is to find files with a specific extension...

find ${DSJobs} -name "*.dsx"

Want to do this for several directories?

for d in <some list of directories>; do
    find ${d} -name ""*.dsx"
done

Want to do something interesting with the files?

find ${DSJobs} -name "*.dsx" -exec dostuffwith.sh "{}" \;