Shell脚本中的psql循环

Question

我需要从file.csv 2值中读取，然后在PostgreSQL中进行13.000个以上的查询。

正如您在上面的代码中看到的那样，这应该是一个非常简单的任务，但是存在一些问题。

#!/bin/bash


MSISDN=($(head file.csv | awk -F ";" '{print $1}' | sed -e "s/^/55/"))
APPID=($(head file.csv | awk -F ";" '{print $3}'))

NUMBER_OF_LINES=$(wc -l file.csv| grep -o "[0-9]*")

for i in $(seq 0 "$NUMBER_OF_LINES")
do

    export PGPASSWORD='MY_PASSWORD'
    psql -q -A -h VERY-LONG-HOST -U MYUSER -d DATABASE -p 1111 -t -c "select 'http://API-HOST/subscription/cancel?subscriptionId=' + s.subscription_id + '&phone=' + s.phone + '&enabled=0&statusId=7&notifyActionListeners=false&extraInfo=TICKET_NUMBER' from sbs.subscription s (nolock) join sbs.configuration c on s.configuration_id = c.configuration_id where c.application_id = ${APPID[$i]} and c.carrier_id = 2 and s.phone = ${MSISDN[$i]};"

done

代码执行时出现错误：

 ERROR: syntax error at or near "112940676229" LINE 2: and c.carrier_id = 2 and s.phone = 55112940676229; 

我如何处理多个查询并在进行另一个查询之前关闭每个查询中的连接，到目前为止如何解决前面显示的错误？

file.csv中内容的示例：

112940676229;Sevice;333                                               
113429402012;Sevice;929  
111429402013;Sevice;888
11240672940;Sevice;445
11320294034;Sevice;333
11429294056;Sevice;22
11942940281;Sevice;122
11962940895;Sevice;233

Answer 1

我建议您尝试这种方法：

create table file_csv (tel bigint, dk text, apid int);
copy file_csv from file.csv delimiter ';';

然后在查询中加入它。

更新感谢@Charles Duffy

如果意图是暂时的，则可以将其设置为临时表（ create temporary table file_csv (tel bigint, dk text, apid int) ）。

Answer 2

为了安全和性能，我强烈建议为此使用其他语言。 也就是说，要回答这个狭窄的问题：

while IFS=';' read -r msidn _ appid _ <&3; do
  appid=${appid%$'\r'} # fix DOS newlines, should they be present in your input
  psql -q -A \
    -h VERY-LONG-HOST -U MYUSER -d DATABASE -p 1111 -t \
    -c "select 'http://API-HOST/subscription/cancel?subscriptionId=' + s.subscription_id + '&phone=' + s.phone + '&enabled=0&statusId=7&notifyActionListeners=false&extraInfo=TICKET_NUMBER' from sbs.subscription s (nolock) join sbs.configuration c on s.configuration_id = c.configuration_id where c.application_id = $appid and c.carrier_id = 2 and s.phone = $msidn;"
done 3< file.csv

---

也就是说，一个不太友好的Python实现可能看起来像：

#!/usr/bin/env python

import csv
import sys
import psycopg2

query='''
select
  s.subscription_id,
  s.phone
from
  sbs.subscription s (nolock)
  join sbs.configuration c on s.configuration_id = c.configuration_id
where
  c.application_id = %(appid)s
  and c.carrier_id = 2
  and s.phone = %(msidn)s
'''

# user is expected to pass CSV file name as our first argument
filename = sys.argv[1]

# user is expected to pass a PostgreSQL connect string as our second argument
conn = psycopg2.connect(sys.argv[2])

curs = conn.cursor()

with open(filename, 'rb') as csv_file:
  csv_reader = csv.reader(csv_file, delimiter=';')
  for csv_row in csv_reader:
    msidn = csv_row[0]
    appid = csv_row[2]
    curs.execute(query, {"msidn": msidn, "appid": appid})
    for result_row in curs.fetchall():
      sid = result_row[0]
      phone = result_row[1]
      print 'http://API-HOST/subscription/cancel?subscriptionId=' + sid + '&phone=' + phone + '&enabled=0&statusId=7&notifyActionListeners=false&extraInfo=TICKET_NUMBER'