Fastest way to merge a directory tree

Question

I have multiple directories of the form

foo/bar/baz/alpha_1/beta/gamma/files/uniqueFile1
foo/bar/baz/alpha_2/beta/gamma/files/uniqueFile2
foo/bar/baz/alpha_3/beta/gamma/files/uniqueFile3

What is the fastest way to merge these directories to a single directory structure like

foo/bar/baz/alpha/beta/gamma/files/uniqueFile1...uniqueFile3

I could write a python script to do that but is there a faster way to do that on a debian machine ? Can rsync help in this case ?

EDIT:

Apologies for not making it clear earlier, the depth in the examples is ~10-12 and I do not know the some directory names such as alpha*, these are randomly generated while throwing out logs. I was using find with wildcards to list these files earlier but now another level has been added in the path, that caused my find queries to take over a minute from 0.004s. So I am looking for a faster solution.

/known_fixed_path_5_levels/*/known_name*/*/fixed_path_2_levels/n_unique_files

has become

/known_fixed_path_5_levels/*/known_name*/*/xx*/fixed_path_2_levels/unique_file_1
/known_fixed_path_5_levels/*/known_name*/*/xx*/fixed_path_2_levels/unique_file_2
.
.
/known_fixed_path_5_levels/*/known_name*/*/xx*/fixed_path_2_levels/unique_file_n

I basically want to collect all those unique files into one place like how it was before.

Answer 1

With find :

mkdir --parents foo/bar/baz/alpha/beta/gamma/files; #create target directory if nessessary
find foo/bar/baz/alpha_[1-3]/beta/gamma/files -type f -exec cp {} foo/bar/baz/alpha/beta/gamma/files \;

Answer 2

As question is not clear about copying or moving , there is two ways, without copy! Even second part don't effectively copy your data!

Simple bash command

Simply:

cd foo/bar/baz
mv -it alpha/beta/gamma/files alpha_*/beta/gamma/files/uniqueFile*

with -i switch to prevent overwritting.

This will work perfectly for small bunch of files.

More robust and adaptive find syntax

Or by using find:

cd foo/bar/baz
find alpha_* -type f -mindepth 3 -exec mv -it alpha/beta/gamma/files {} +

Advantage of using find are

• you could add a lot of flags like -name , -mtime and so on
• find will never try to pass more files to command ( mv ) that command line could hold.

cp -al specific UN*X concept

Under Un*x, you could create hard-link wich is not symbolic links, but a secondary entry in folder tree, for the same inode.

Nota: As only one inode has to be referenced, this could work only on same filesystem.

By using

cp -ialt alpha/beta/gamma/files alpha_*/beta/gamma/files/uniqueFile*

You will copy in one directory all inodes references, but keeping only one file for each.

Using bash 's globstar feature:

cd foo/bar/baz
shopt -s globstar
cp -alit alpha/beta/gamma/files alpha_*/**/uniqueFile*