[英]How to get psycopg2's description from PostgreSQL server side cursor
I built a class that gets an arbitrary Postgres SQL query, fetch data and creates a CSV file.我构建了一个获取任意 Postgres SQL 查询、获取数据并创建 CSV 文件的类。 I'm using
cursor.description
to get column names, passing it as my csv header.我正在使用
cursor.description
来获取列名,并将其作为我的 csv 标题传递。 However data sets got too large and I'm moving to server side cursors.但是数据集太大了,我正在转向服务器端游标。
Server Side cursors doesn't seem to have any data under description.服务器端游标似乎没有任何描述数据。 When I run:
当我运行时:
import psycopg2
conn = psycopg2.connect(**conn_info)
cursor = conn.cursor("server_side")
cursor.execute("select * from foo")
print(cursor.description)
It prints None
, probably because query didn't actually ran.它打印
None
,可能是因为查询实际上没有运行。 But is there a way to get column names in this configuration?但是有没有办法在这个配置中获取列名?
The query in cursor.execute('select...')
is executed on the server side but the application has no data yet, hence cursor.description
is undefined. cursor.execute('select...')
中的查询在服务器端执行,但应用程序还没有数据,因此cursor.description
未定义。 To get the description you need to get at least a row from the server-side cursor, eg:要获取描述,您需要从服务器端游标中至少获取一行,例如:
cursor = conn.cursor("server_side")
# or
# cursor = conn.cursor("server_side", scrollable=True)
# see below
cursor.execute("select * from my_table")
first_row = cursor.fetchone()
print(cursor.description)
# you can place the cursor in the initial position if needed:
# cursor.scroll(-1)
Note that you won't get the description when the table is empty.请注意,当表格为空时,您将无法获得描述。
There is no better (faster or simpler) way to get a query result description of a named cursor.没有更好(更快或更简单)的方法来获取命名游标的查询结果描述。 This is due to the way named cursors are implemented.
这是由于命名游标的实现方式。 The commands
命令
cursor = conn.cursor("server_side")
cursor.execute("select * from my_table")
are implemented by declaring a cursor using the Postgres command:通过使用 Postgres 命令声明游标来实现:
DECLARE "server_side" CURSOR WITHOUT HOLD FOR select * from my_table
Per the documentation:根据文档:
DECLARE allows a user to create cursors, which can be used to retrieve a small number of rows at a time out of a larger query.
DECLARE 允许用户创建游标,可用于从较大的查询中一次检索少量行。 After the cursor is created, rows are fetched from it using FETCH.
创建游标后,使用 FETCH 从中获取行。
The cursor declaration itself does not give any information on the structure of the results.游标声明本身不提供有关结果结构的任何信息。 We can obtain it only after fetching a row or rows by the
FETCH
command.我们只有在通过
FETCH
命令获取一行或多行后才能获取它。
The other answers here are;这里的其他答案是; unfortunately, the answer and here's why.
不幸的是,答案和原因如下。 There is no way to get the
description
or even rowcount
back from a server-side cursor without first invoking a fetch
.如果不首先调用
fetch
,就无法从服务器端游标中获取description
甚至行rowcount
。 It returns None
as per PEP-249 :它根据PEP-249返回
None
:
This attribute will be
None
for operations that do not return rows or if the cursor has not had an operation invoked via the .execute*() method yet.对于不返回行的操作或游标尚未通过.execute*()方法调用的操作,此属性将为
None
。
This is because even though you've called execute
, the server may not have yet executed the query and we can confirm that by checking the logs (where logging is set to all
)这是因为即使您调用了
execute
,服务器可能还没有执行查询,我们可以通过检查日志来确认(其中 logging 设置为all
)
Using the following code with a 30-second sleep for clarity为了清楚起见,使用以下代码和 30 秒睡眠
cursor = conn.cursor("server_side")
cursor.execute("select * from foo")
time.sleep(30)
cursor.fetchall()
print(cursor.description)
The logs will show日志将显示
2020-06-19 12:11:37.687 BST [11916] LOG: statement: BEGIN
2020-06-19 12:11:37.687 BST [11916] LOG: statement: DECLARE "server_side" CURSOR WITHOUT HOLD FOR select * from foo
2020-06-19 12:12:07.693 BST [11916] LOG: statement: FETCH FORWARD ALL FROM "server_side"
Notice the 30~ second gap between the declaration and the FETCH
, the latter being the invocation that allows us to get the description
from the cursor.注意声明和
FETCH
之间的 30~ 秒间隔,后者是允许我们从光标获取description
的调用。
Without server_side
for comparison没有
server_side
进行比较
2020-06-19 12:11:01.310 BST [3012] LOG: statement: BEGIN
2020-06-19 12:11:01.311 BST [3012] LOG: statement: select * from foo
Your only options are to use scroll
or perform a LIMIT 1
select prior to your larger query.您唯一的选择是在更大的查询之前使用
scroll
或执行LIMIT 1
选择。
A less attractive option is to use the INFORMATION_SCHEMA
table like so一个不太吸引人的选择是像这样使用
INFORMATION_SCHEMA
表
select column_name, data_type, character_maximum_length
from INFORMATION_SCHEMA.COLUMNS where table_name = 'foo';
Could you not do:你不能这样做:
import psycopg2
conn = psycopg2.connect(**conn_info)
cursor_desc = conn.cursor()
cursor_desc.execute("select * from foo limit 1")
print(cursor_desc.description)
cursor = conn.cursor("server_side")
cursor.execute("select * from foo")
Then you are not messing with the server side data returning query.那么您就不会弄乱服务器端数据返回查询。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.