Psycopg2 query remove \n from args

Question

Im using python3 and postgres 11.5.

This is the script :

a = cursor.execute("SELECT tablename FROM pg_catalog.pg_tables limit 5")
for table in a:
    cursor.execute("SELECT * FROM pg_prewarm(public.%s)", [table[0]])

a query gets some table names , and the loop query should run table name as the %s. but for some reason i get the arg table[0] with // /n in the query and its messing it up.

if i print a results i get table names as tuple:

[('sa1591354519',), ('sa1591397719',), ('sa1591397719',)]

so [table[0]] is a string.

the error i get:

1574683839 [16177], ERR, execute ({'Error while connecting to PostgreSQL': SyntaxError('syntax error at or near "\'sa1591440919\'"\nLINE 1: SELECT * FROM pg_prewarm(public.\'sa1591440919\')\n                                        ^\n')},)

what can i do ?

Answer 1

The errors don't have anything to do with the newlines you see, which are just an artifact of the error message. If you were to print out the error, would see:

syntax error at or near "'sa1591440919'"
LINE 1: SELECT * FROM pg_prewarm(public.'sa1591440919')
                                        ^

In other words, Postgres doesn't like the table name you're passing because it contains quotes. This is happening because you're trying to treat the table names like a normal query parameter, which causes psycopg to quote them...but that's not what you want in this case.

Just replace your use of query templating with normal Python string substitution:

a = cursor.execute("SELECT tablename FROM pg_catalog.pg_tables limit 5")
for table in a:
    cursor.execute("SELECT * FROM pg_prewarm(public.%s)" % (table[0]))

But this won't actually work, because cursor.execute doesn't return a value, so a will be None . You would need to do something like:

cursor.execute("SELECT tablename FROM pg_catalog.pg_tables limit 5")
a = cursor.fetchall()
for table in a:
  ...