在一个 psql 复制调用中将多个 csv 文件复制到 postgresql 表

Question

loading a csv file to my postgresql db table using psql utility copy as below;

psql -h $host -p $port -d $db -U $username -c "\copy \"$schema\".\"$table\" from '$datafile' with delimiter as ',' csv"

this works great. However, I need to load many files at the same time. I can run above psql copy command in parallel fashion. Since each call starts a new process (my app is do.net core 2.1 and is running on openshift), increasing the number of file also increases the memory and CPU. Process time also increases. What I actually need is to call psql copy with multiple csv file. I need to create the parallelization on the db, not on the openshift pod.
How can I do this? If this is not possible, what is the most optimum way to load multiple files at a time?

Thankx

Answer 1

cat *.csv | psql -h $host -p $port -d $db -U $username -c "\copy \"$schema\".\"$table\" from STDIN with delimiter as ',' csv"