How to exclude directories from xargs rsync?

Question

So I'm trying to set up a bash script for incremental backups using rsync with xargs to use multiple threads on Ubuntu Server 20.042 LTS. There's a handful of directories that I want to exclude, but I can't seem to figure out the way to do that. Currently I'm piping ls to xargs which then executes rsync with 8 processes:

# Create the archive and begin the backup.
cd /
ls --indicator-style none \
  --ignore={"bin","boot","cdrom","data","dev","init*","lib","lib64","lost+found"} \
  --ignore={"media","mnt","proc","root","run","sbin","snap","srv","swap.img","sys"} \
  --ignore={"tmp","vmlinuz*"} | xargs --max-args 1 --max-procs 8 -I % \
  rsync --archive --compress --delete --ignore-missing-args --recursive --verbose \
  --link-dest "${backup_link}" "%" \
  --exclude={"*tmp*","*.cache"} \
  "${backup_path}"

For some reason, it's still trying to use rsync on the directories that I've ignored with the ls command. I've tried setting rsync to exclude the directories instead:

# Create the archive and begin the backup.
cd /
ls --indicator-style none | xargs --max-args 1 --max-procs 8 -I % \
  rsync --archive --compress --delete --ignore-missing-args --recursive \
  --exclude={"/bin","/boot","/cdrom","/data/mysql","/data/backup","/dev"} \
  --exclude={"/initrd.img","/initrd.img.old","/lib","/lib64","/lost+found","/media"} \
  --exclude={"/mnt","/proc","/root","/run","/sbin","/snap","/srv","/swap.img","/sys"} \
  --exclude={"**/tmp","/vmlinuz*","*.cache"} \
  --link-dest "${backup_link}" "%" "${backup_path}"

Which still tries to backup the files that I want to exclude.

The reason I'm trying to run multiple threads is that this script is set to execute every 15 minutes via crontab and, without the multiple processes, it's barely taking less than 15 minutes. Later in the script, it then copies that backup (also using rsync ) to an SMB server, which is a huge bottleneck with all of the small files.

Is there something that I'm missing here? Or is there a better way to accomplish this? This is a production server, so I'd really like to avoid installing third-party utilities.

Thank you in advance!

Answer 1

EDIT : in my first answer, I indicated that your syntax for the --ignore option did not work on Mint 19 (ls GNU 8.28). Well it does, so this part is now removed!

---

I always try to use find for such tasks, and not trust ls . You can use this:

find / -maxdepth 1 -type d ! -name "bin" -print

Use ! -name "XXX" ! -name "XXX" sections many times for your exclusions.

---

Or my preferred method, since I find it easier to read:

find / -maxdepth 1 -type d -print | egrep -Ev "bin|boot"

You can have as many patterns inside the egrep match, separated by a | . In this context, | is like or . So here it says to filter out "bin" or "boot".