Python + Postgres + psycopg2 : Composing a dynamic json query and dealing with quotes

Question

I'm trying to compose a dynamic query involving a json column in Postgres using Python and the psycopg2 driver.

where_clause = ''
parameters = []

for key, v in (arguments):
    parameters.append(key)
    parameters.append(v)
    where_clause = where_clause + "data @> '{\"%s\":%s}' AND "

sql = u'select * from my_table where ' + where_clause + ' 1 = 1
db = connect(connection_string)
cur = db.cursor()
cur.execute(sql, tuple(parameters))
rows = cur.fetchall()
cur.close()
db.close()

When I do this, it has a problem with this '{\\"%s\\":%s}' .

psycopg2.ProgrammingError: syntax error at or near "favorite_topping"
LINE 1: select * from my_table where data @> '{"'favorite_topping'":'pepperoni'...

It doesn't like how I've arranged the quotes.

The problem appears to be how psycopg2 is chosing to insert quotes when it composes the final query.

Update

If I do this: where_clause = where_clause + "data @> \\"{%s:%s}\\" AND "

what I get is this:

where data @> "{'favorite_topping':'pepperoni'}"

and that fails because what Postgres wants are the quotes to be the other way around, like so:

where data @> '{"favorite_topping":"pepperoni"}'

Is there any way to get that to happen?

Answer 1

This is how the query parameterization works - the database driver determines the type of a query parameter to be a string and automatically puts quotes around the escaped value.

You can workaround it by using regular string formatting , but make sure to validate/sanitize/escape the query parameters yourself to avoid SQL injection attacks and problems with balanced quotes.

Answer 2

This should solve your problem:

from psycopg2.extras import Json
from psycopg2.extras import register_default_json
register_default_json(loads=lambda x: x)

query = u'select * from my_table where data @> %s;'
cur.execute(sql, Json({'favorite_topping':'pepperoni'})

Based largely on information in https://github.com/psycopg/psycopg2/issues/190 .