如何将 Find 的 output 排序到 psql 复制命令以按顺序加载数据？

Question

I wish to load data to a PostgreSQL DB from a bunch of files that are in multiple folders. I have to load them in order (ie files in folders 2020 must be loaded before folders 2021 and so on). This is what I have currently:

find  ~/data/inserts/ -type f -exec psql -h db1.cluster-xxxxx.us-east-1.rds.amazonaws.com -p 5432 dbname -U admin1 -c "\COPY public.db1(col1,col2) FROM '{}' DELIMITER ',' CSV HEADER;" \;

This loads the data in the files, but the files aren't sorted. By Googling, I know you can pipe into sort like so:

find ~/data/inserts/ -type f -print | sort -z | xargs -r0 echo

but I am not sure how to apply it to my case. I am not sure how to use xargs -r0 even after reading the docs.

Answer 1

You need -print0 instead of -print as find argument:

#!/usr/bin/env bash

# Pipe the sorted null delimited output of find to while loop
find ./ -type f -print0 | sort -z |
while IFS= read -r -d '' input_file || [ -n "$input_file" ]; do
  # Now execute the pgsql command to copy from STDIN rather than named file
  psql \
    -h db1.cluster-xxxxx.us-east-1.rds.amazonaws.com -p 5432 -U admin1 dbname \
    -c "COPY public.db1(col1,col2) FROM STDIN DELIMITER ',' CSV HEADER;" \
    <"$input_file" # This provide the input file as STDIN
done