Why the Linux command CP behave differently in CLI and in script?

Question

I want to copy a bunch of Verilog/systemverilog sources, so I use CP with wildcard expression:

cp <some_dir>/*.{v,sv,svh} .

It works. But when I put it to a script with exactly the same line, the CP command fails with the log:

cp: cannot stat `../../mytest/spiTest/*.{v,sv,svh}': No such file or directory

How is that happening?

PS: I use bash as the shell.

---

And here is my script:

#!/bin/bash
rdir=../../mytest/spiTest
f1="$rdir/bench.lst"
f2="$rdir/cphex" #the script to copy rom data
f3="$rdir/make*" #makefile scripts
f4="$rdir/*.hex" #rom files
f5="$rdir/*.{v,sv,svh}" #testbench files
echo 'Copying files...'
cp $f1 $f2 $f3 $f4 .
cp $f5 .

---

I do changed the first line to

#!/bin/bash -vx

and run this script again, and I get:

#!/bin/bash -vx

rdir=../../mytest/spiTest
+ rdir=../../mytest/spiTest
f1="$rdir/bench.lst"
+ f1=../../mytest/spiTest/bench.lst
f2="$rdir/cphex" #the script to copy rom data
+ f2=../../mytest/spiTest/cphex
f3="$rdir/make*" #makefile scripts
+ f3='../../mytest/spiTest/make*'
f4="$rdir/*.hex" #rom files
+ f4='../../mytest/spiTest/*.hex'
f5="$rdir/*.{v,sv,svh}" #testbench files
+ f5='../../mytest/spiTest/*.{v,sv,svh}'

echo 'Copying files...'
+ echo 'Copying files...'
Copying files...
cp $f1 $f2 $f3 $f4 .
+ cp ../../mytest/spiTest/bench.lst ../../mytest/spiTest/cphex ../../mytest/spiTest/makefile ../../mytest/spiTest/makefile.defines ../../mytest/spiTest/rom.hex ../../mytest/spiTest/rom_if.hex .
cp $f5 .
+ cp '../../mytest/spiTest/*.{v,sv,svh}' .
cp: cannot stat `../../mytest/spiTest/*.{v,sv,svh}': No such file or directory

Answer 1

Check the first line of the script. It probably reads:

#!/bin/sh

which switches the shell from BASH to Bourne Shell. Use

#!/bin/bash

instead.

[EDIT] You're running into problems with expansion. BASH has a certain order in which it expands patterns and variables. That means:

f5="$rdir/*.{v,sv,svh}" #testbench files

is quoted, so no file name expansion happens at this time. Only the variable $rdir is expanded. When

cp $f5 .

is executed, BASH first looks for file names to expand and there are none. Then it expands variables ( f5 ) and then calls cp with two arguments: ../../mytest/spiTest/*.{v,sv,svh} and . . Since cp expects the shell to have performed the file name expansion already, you get an error.

To fix this, you have to use arrays:

f5=($rdir/*.{v,sv,svh})

This replaces the variable and then expands the file names and puts everything into the array f5 . You can then call cp with this array while preserving whitespaces:

cp "${f5[@]}" .

Every single character here is important. [@] tells BASH to expand the whole array here. The quotes say: Preserve whitespace. {} is necessary to tell BASH that [@] is part of the variable "name" to expand.

Answer 2

Here's the problem: the order of substitutions. Bash performs brace expansion before variable expansion. In the line cp $f5 . , bash will do:

1. brace expansion: n/a
   • this is the key point: the variable contains a brace expression, but the shell does not see it now when it needs to.
2. tilde expansion: n/a
3. parameter expansion: yes -- cp ../../mytest/spiTest/*.{v,sv,svh} .
4. command substitution: n/a
5. arithmetic expansion: n/a
6. process substitution: n/a
7. word splitting: n/a
8. filename expansion: yes, bash looks for files in that directory ending with the string
   .{v,sv,svh} . It finds none, nullglob is not set, thus the pattern is not removed from the command
9. quote removal: n/a

Now the command is executed and fails with the error you see.

https://www.gnu.org/software/bash/manual/bashref.html#Shell-Expansions

Solutions:

1. use Aaron's idea of an array
2. (not recommended) force a 2nd round of expansions: eval cp $f5 .

Answer 3

The line

 f5="$rdir/*.{v,sv,svh}" #testbench files

is probably wrong. First, avoid comments at end of line, they should be (at least for readability) in a separate line. Then, avoid using globbing in variable assignment. So remove that line, and code later (that is, replace the old cp $f5 . line with)

 cp "$rdir"/*.{v,sv,svh} .

BTW, I would test that "$rdir" is indeed a directory with

 if [ ! -d "$rdir" ] ; then
    echo invalid directory $rdir > /dev/stderr
    exit 1
 fi

You should read the Advanced Bash Scripting Guide