使用tar压缩目录时如何排除大文件

Question

I want to compress a directory in Linux. I created a tar.gz that it turns to be a big file, due to the reason that the directory contains some *.o files and some pdf files.

Is there any way to compress a directory but exclude files larger than a predefined SIZE? There is a --exclude argument in tar command, however I would like to reject files larger than 1 MB. This is the constrain, not the name of the file.

Answer 1

Based on Jan-Philip Gehrcke's response:

find . -type f -size -1024k -print0 | tar -czf --null -T - -f archive.tar.gz

for files less than 1M. Tested on OS X and Ubuntu Linux.

Answer 2

The ...| tar c --null -T - ...| tar c --null -T - solution above is the best if you have adequate memory (ie the file list fits into your memory easily (in most cases, this is true)). However, xargs does have a place if you are memory-constrained, but you have to use it appropriately so that the multiple tar invocations have no ill effect.

To compress, you may use:

find . -type f -size -1024k | xargs tar c | gzip > archive.tar.gz

This results in a file of concatenated tar archives, gzipped together into the resulting file (you may also use cz and omit | gzip as concatenating gzip archives is still valid gzip, but you lose a tiny bit of compression, or quite a bit of compression if you use bzip2 or xz instead of gzip).

To extract the resulting file you have to use the --ignore-zeros or -i option of tar to not only extract the first archive:

tar xizf archive.tar.gz

Answer 3

You could use a combination of find (with its -size flag) and xargs to pass it into tar.

Something like:

find . -size -100k -print | xargs tar cvf archive.tar

for files less than 100k. See man find for the other size options

Answer 4

find ./myRep/ -type f -size -1024k | xargs tar cfvz myArchive.tar

In a word, first part of this expression construct a list of files that size is lower than 1024k recursively from ./myRep/ and second part create tar/gzip archive.