将多个目录中的多个文件从服务器复制到本地，文件进入与服务器同名的目录

Question

Sorry if the title is confusing

Basically, on my server, I have multiple files for multiple runs of an analysis for multiple traits, eg:

/Weight/run1/file1.txt
/Weight/run1/file2.txt
/Weight/run2/file1.txt
/Weight/run2/file2.txt
/LegLength/run1/file1.txt
/LegLength/run1/file2.txt
/LegLength/run2/file1.txt
/LegLength/run2/file2.txt

In total I have 11 traits with two runs each, with each run generating 26 files.

I want to copy them across onto my local machine, but keeping the file hierarchy the same. I can only find code that copies from multiple directories into one directory, which is not what I want.

So far I've managed to hack together the following code from somewhat related stackoverflow questions, though obviously doesn't work and just tries to find ~/Documents/path/to/local/directories on the server. I was wondering if anyone had any advice on how to tweak it?

`ssh user@server 'for f in /path/to/server/directories/*/*/*txt ; do newpath="${f%/*}"; newpath=${newpath:41}  ; scp user@server:${f} ~/Documents/path/to/local/directories/${newpath};  done'`

I also have multiple other files in each end directory on the server with different extensions, but I am not interested in copying them across.

(Edit to add: I'm using newpath=${newpath:41} because it removes /path/to/server/directories/ from the variable)

Answer 1

It's a little complex to get it right but a combination of --include and --exclude in the rsync command should be able to do what you want:

rsync -av \
      --include '/Weight/' \
      --include '/Weight/run[12]/' \
      --include '/Weight/run[12]/file[12].txt' \
      --include '/LegLength/' \
      --include '/LegLength/run[12]/' \
      --include '/LegLength/run[12]/file[12].txt' \
      --exclude '/*' \
      --exclude '/*/*' \
      --exclude '/*/*/*' \
      user@server:'/path/to/server/directories/' ./here/

^{note: in this case, the first / in the includes & excludes means /path/to/server/directories/}