Running commands in subdirectories with bash

Question

I have a sequence of directories that I need to run various shell commands on and I've made a short script called dodirs.sh to simplify running the command in each directory:

#!/bin/bash
echo "Running in each directory: $@"
for d in ./*/; do
    (
    cd "$d"
    pwd
    eval "$@"
)
done

This is fine for many simple commands, but some have trouble, such as:

grep "free  energy   TOTEN" OUTCAR | tail -1

which looks for a string in a file located in each directory.

It seems that the pipe and/or the quotes is the trouble since if I say:

dodirs.sh grep "free  energy   TOTEN" OUTCAR

I get a sensible (if waaaay to long output) along the lines of:

Running in each directory: grep free  energy   TOTEN OUTCAR
...
OUTCAR:  free energy    TOTEN  =      -888.53122906 eV
OUTCAR:  free energy    TOTEN  =      -888.53132396 eV
OUTCAR:  free  energy   TOTEN  =      -888.531324 eV
...

I notice the result of the echo loses the quotes, so that is a bit odd. On the other hand, if I say:

dodirs.sh grep "free  energy   TOTEN" OUTCAR | tail -1

then I get the nonsensical:

...
grep: energy: No such file or directory
grep: TOTEN: No such file or directory
...

Notice the echo doesn't echo at all now and it is clearly misinterpreting the line.

Is there some way I have to escape characters, or package the parameters inside my dodirs.sh script?

And maybe someone knows of a better approach altogether?

Answer 1

The quotes disappear because they aren't necessary once the shell identifies the words to pass to your script as arguments. Inside your script, $1 is grep , $2 is free energy TOTEN , etc.

You do need to escape the pipe (with a backslash \\| or by quoting '|' ), though, so that it also is passed as an argument to eval .

dodirs.sh grep "free  energy   TOTEN" OUTCAR \| tail -1

Answer 2

Consider:

#!/bin/bash

# use printf %q to generate a command line identical to what we're actually doing
printf "Running in each directory: " >&2
printf '%q ' "$@" >&2
echo >&2

# use && -- we don't want to execute the command if cd into a given directory failed!
for d in ./*/; do
    (cd "$d" && echo "$PWD" >&2 && "$@")
done

This is much more predictable: It passes exact argument lists through, so for a general command you can just quote it naturally. (This is the exact same behavior as you get with find -exec or other tools which call execv* -family calls with a literal, passed-through argument list; thus, it means you get identical behavior to sudo , chpst , chroot , setsid , etc).

For a single command, invocation looks like what you'd expect:

dodirs grep "free  energy   TOTEN" OUTCAR

To execute shell directives, such as pipelines, explicitly execute a shell:

dodirs sh -c 'grep "free  energy   TOTEN" OUTCAR | tail -n 1'
#      ^^ ^^

...or, if you're willing to let callers rely on implementation details (such as the fact that this is implemented with a shell, and exactly which shell it's implemented with), use eval :

dodirs eval 'grep "free  energy   TOTEN" OUTCAR | tail -n 1'
#      ^^^^

This may be slightly more work, but it puts you in line with standard UNIX conventions, and avoids risking shell injection vulnerabilities if callers fail to quote their arguments to be eval -safe.