Rsync使用find命令删除原始文件，用符号链接替换原文

Question

I work for a graphics company, and our clients submit massive amounts of data. A single project folder can take up hundreds of gigabytes of space on our server. While space on my direct attached RAID is limited to 10TB, I have a 96TB storage server connected to my main server via iSCSI.

I'm not versed enough in BASH scripting to automate this process, but what I do manually is perform a find with this command:

find /disk -type d -name 09\ Client-Files | sed 's/ /\\ /g' > client-origs.txt

From this list I rsync the "09 Client-Files" directory to the storage server using:

rsync -avR /disk/Client-Jobs/1234\ Client/09\ Client-Files /iscsi

With the -R option it's copying the relative path to /iscsi.

After this the original "09 Client-Files" directory is deleted and a symlink pointing to its new location on the /iscsi volume.

What I'm shooting for is to use the found path as a variable to do everything at once. I'm really at a loss on how to proceed. I've been trying to educate myself on bash variables for days now — I can get them to work on simpler things but in this case I'm coming up empty. Can you help a brother out?

Answer 1

have you looked at the lndir command? (may need to be installed separately)

It creates a shadow directory of symbolic links pointing to another directory tree.

Answer 2

You can do all that at once with find:

find /usr -type d -name My\ Files -exec rsync -avR '{}' /iscsi \;

For more info man find , right before the end there are several examples. Alternatively you can pipe the results using the xargs utility.

You should explore your manuals more througly, there is a lot of treasure to be found.

On a bash script you could also put:

export MIVARIABLE=$(find /usr -type d -name My\ files)

and then

for i in $MIVARIABLE ;do
   rsync -avR $i /iscsi;
done

But this has drawbacks among them if your results are very big, it may exceed bash maximum text space available for a variable , and if the retrieved names include spaces, some utilities may go crazy.

In a modern system the maximum size on a bash environment variable can be from 8kib to nearly 2MiB, if you put to much info on a variable you'll get a Argument list too long error.

So if you need to iterate over a file/directory list you better use find built-in functions or in cooperation with xargs . Read carefully their manuals and try some dry runs using echo your_command_line instead, so you gain appropiate experience and confidence with the utilities. Look for the -print0 function on find as well as the -0 command on xargs which are designed to avoid some problems with filenames with spaces and symbols in their names.

EDIT: find will handle white spaces correctly with the -exec operator:

find /usr -type d -name My\ Files -exec rsync -avR '{}' /iscsi \;  -exec rm -rf '{}'\;

would do what you need one after the other. I was pointing out options, so you could decide what was better suited to your needs, I did not meant to confuse you.

Edit2:

find /usr -type d -name 'My Files' -exec rsync -avR '{}' /iscsi \;  -exec rm -rf '{}'\;

funciona igual que el ejemplo de arriba. Cuando bash lee 'My Files' it takes the whole word togheter and does not treat the white spaces inside differently