如何查找两台计算机上的文件/文件夹？

Question

I have a folder called documentaries on my Linux computer.
I have SSH access to seedbox (also Linux).

How do I find out which documentaries I have in both computers?

On seedbox it's a flat file structure. Some documentaries are files, some are folders which contain many files, but all in same folder For example:

data/lions_botswana.mp4
data/lions serengeti/S01E01.mkv
data/lions serengeti/S01E02.mkv
data/strosek_on_capitalism.mp4
data/something_random.mp4

Locally structure is more organized

documentaries/animals/lions_botswana.mp4
documentaries/animals/lions serengeti/S01E01.mkv
documentaries/animals/lions serengeti/S01E02.mkv
documentaries/economy/strosek_on_capitalism.mp4
documentaries/something_random.mp4

I am not looking for command like diff , I am looking for command like same (opposite of diff) if such command exists.

Answer 1

Based on the answer from Zumo de Vidrio, and my comment:

on one computer

  cd directory1/; find | sort > filelist1

on the other

  cd directory2/; find | sort > filelist2

copy them in one place an run:

  comm -12 filelist1 filelist2

or as a one liner:

ssh user@host 'cd remotedir/; find|sort' | comm -12 - <(cd localdir/; find|sort)

Edit: With multiple folders this would look as follows

on one computer

  cd remotedir/; find | sort > remotelist

on the other

  cd localdir/subdir1/; find > locallist1
  cd -;
  cd localdir/subdir2/; find > locallist2
  cd -;
  #... and so on
  sort locallist1 locallist2 > locallistall

copy them in one place an run:

  comm -12 remotelist locallistall

or as a (now very long) one liner:

ssh user@host 'cd remotedir/; find|sort' | comm -12 - <({cd localdir/subdir1/; find; cd -; cd localdir/subdir2/; find; cd -; cd localdir/subdir3/; find}|sort)

Answer 2

Export list of remote files to local file by:

ssh user@seedbox 'find /path/to/data -type f -execdir echo {} ";"' > remote.txt

^{Note: On Linux you've to use absolute path to avoid leading ./ or use with "$PWD"/data .}

Then grep the result of find command:

find documentaries/ -type f | grep -wFf remote.txt

This will display only these local files which also exist on remote.

If you would like to generate similar list on local and compare two files, try:

find "$PWD"/documentaries/ -type f -execdir echo {} ';' > local.txt
grep -wFf remote.txt local.txt

However above methods aren't reliable, since one file could have a different size. If files would have the same structure, you could use rsync to keep your files up-to-date.

---

For more reliable solution, you can use fdupes which can find all files which exist in both directories by comparing file sizes and MD5 signatures.

Sample syntax:

fdupes -r documentaries/ data/

However both directories needs to be accessible locally, so you can always use sshfs tool to mount the remote directory locally. Then you can use fdupes to find all duplicate files. It has also option to remove the other duplicates ( -d ).

Answer 3

Copy the ls output of each Computer to a same folder and then apply diff over them:

In your computer:

ls -R documentaries/ > documentaries_computer.txt

In seedbox:

ls -R documentaries/ > documentaries_seedbox.txt

Copy both files to a same location and execute:

diff documentaries_computer.txt documentaries_seedbox.txt

Answer 4

You can mount remote folder using sshfs , then you can use diff -r to find the differences between them.

Eg

sshfs user@seedbox-host:/path/to/documentaries documentaries/
diff -rs /local/path/documentaries/animals documentaries/ | grep identical
diff -rs /local/path/documentaries/economy documentaries/ | grep identical