[英]psql - read SQL file and output to CSV
I have a SQL file my_query.sql
:我有一个 SQL 文件my_query.sql
:
select * from my_table
Using psql
, I can read in this sql file:使用psql
,我可以在这个 sql 文件中读取:
\\i my_query.sql
Or pass it in as an arg:或者将其作为参数传递:
psql -f my_query.sql
And I can output the results of a query string to a csv:我可以将查询字符串的结果输出到 csv:
\\copy (select * from my_table) to 'output.csv' with csv header
Is there a way to combine these so I can output the results of a query from a SQL file to a CSV?有没有办法将这些组合起来,以便我可以将查询结果从 SQL 文件输出到 CSV?
Unfortunately there's no baked-in functionality for this, so you need a little bash-fu to get this to work properly.不幸的是,这里没有内置功能,因此您需要一点 bash-fu 才能使其正常工作。
CONN="psql -U my_user -d my_db"
QUERY="$(sed 's/;//g;/^--/ d;s/--.*//g;' my_query.sql | tr '\n' ' ')"
echo "\\copy ($QUERY) to 'out.csv' with CSV HEADER" | $CONN
The sed
fun removes all semicolons, comment lines, and end of line comments, and tr
converts newlines to spaces (as mentioned in a comment by @abelisto): sed
fun 删除所有分号、注释行和行尾注释, tr
将换行符转换为空格(如@abelisto 的评论中所述):
-- my_query.sql
select *
from my_table
where timestamp < current_date -- only want today's records
limit 10;
becomes:变成:
select * from my_table where timestamp < current_date limit 10
which then gets passed in to the valid psql
command:然后传递给有效的psql
命令:
\copy (select * from my_table where timestamp < current_date) to 'out.csv' with csv header
Here's a script:这是一个脚本:
sql_to_csv.sh
#!/bin/bash
# sql_to_csv.sh
CONN="psql -U my_user -d my_db"
QUERY="$(sed 's/;//g;/^--/ d;s/--.*//g;' $1 | tr '\n' ' ')"
echo "$QUERY"
echo "\\copy ($QUERY) to '$2' with csv header" | $CONN > /dev/null
./sql_to_csv.sh my_query.sql out.csv
You could do it using a bash script.您可以使用 bash 脚本来完成。
dump_query_to_csv.sh: dump_query_to_csv.sh:
#!/bin/bash
# Takes an sql query file as an argument and dumps its results
# to a CSV file using psql \copy command.
#
# Usage:
#
# dump_query_to_csv.sh <sql_query_file> [<csv_output_filesname>]
SQL_FILE=$1
[ -z $SQL_FILE ] && echo "Must supply query file" && exit
shift
OUT_FILE=$1
[ -z $OUT_FILE ] && OUT_FILE="output.csv" # default to "output.csv" if no argument is passed
TMP_TABLE=ttt_temp_table_xx # some table name that will not collide with existing tables
## Build a psql script to do the work
PSQL_SCRIPT=temp.psql
# create a temporary database table using the SQL from the query file
echo "DROP TABLE IF EXISTS $TMP_TABLE;CREATE TABLE $TMP_TABLE AS" > $PSQL_SCRIPT
cat $SQL_FILE >> $PSQL_SCRIPT
echo ";" >> $PSQL_SCRIPT
# copy the temporary table to the output CSV file
echo "\copy (select * from $TMP_TABLE) to '$OUT_FILE' with csv header" >> $PSQL_SCRIPT
# drop the temporary table
echo "DROP TABLE IF EXISTS $TMP_TABLE;" >> temp.sql
## Run psql script using psql
psql my_database < $PSQL_SCRIPT # replace my_database and add user login credentials as necessary
## Remove the psql script
rm $PSQL_SCRIPT
You'll need to edit the psql
line in the script to connect to your database.您需要编辑脚本中的psql
行以连接到您的数据库。 The script could also be enhanced to take the database and account credentials as arguments.还可以增强脚本以将数据库和帐户凭据作为参数。
The accepted solution is correct, but I had Windows and had to make it run via a batch (command) file.接受的解决方案是正确的,但我有 Windows 并且必须通过批处理(命令)文件运行它。 Posting it here if someone needs that如果有人需要,请在此处发布
@echo off
echo 'Reading file %1'
set CONN="C:\Program Files\PostgreSQL\11\bin\psql.exe" -U dbusername -d mydbname
"C:\Program Files\Git\usr\bin\sed.exe" 's/;//g;/^--/ d;s/--.*//g;' %1 | "C:\Program Files\Git\usr\bin\tr.exe" '\n' ' ' > c:\temp\query.txt
set /p QUERY=<c:\temp\query.txt
echo %QUERY%
echo \copy (%QUERY%) to '%2' WITH (FORMAT CSV, HEADER) | %CONN%
我认为最简单的方法是利用shell的变量扩展能力:
psql -U my_user -d my_db -c "COPY ($(cat my_query.sql)) TO STDOUT WITH CSV HEADER" > my_query_results.csv
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.