Python3: ValueError: unsupported format character 'B' (0x42) at index 568

Question

So I'm trying to save a URL to a database but I keep getting this error:

ValueError: unsupported format character 'B' (0x42) at index 568

when I try to execute the query.

The URL:

https://images-na.ssl-images-amazon.com/images/I/81Sp%2BP3kX-L._AC_SL1500_.jpg

The code:

query = f"INSERT INTO Ruby.available_links(user_email, product_name, product_price, product_currency, product_old_price, product_store, user_link, spider_link, spider, created_date, product_img)" \
                    f"VALUES ('{email}', '{product_name}', '{product_price}', '{str(product_currency)}', '{product_price}', '{company[0]}', '{user_short_url['link']}', '{str(url)}', '{spider}', '{str(date.today())}', '{str(product_img)}')"
            conn.execute(query)

I know that is the url causing the problem because when I removed it from the query it successfully executed the query and did not display any error.

Full traceback:

File "new_link_handler.py", line 70, in runner_new_links
    conn.execute(query)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 976, in execute
    return self._execute_text(object_, multiparams, params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1149, in _execute_text
    parameters,
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
    e, statement, parameters, cursor, context
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1478, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python3.6/dist-packages/pymysql/cursors.py", line 168, in execute
    query = self.mogrify(query, args)
  File "/usr/local/lib/python3.6/dist-packages/pymysql/cursors.py", line 147, in mogrify
    query = query % self._escape_args(args, conn)
ValueError: unsupported format character 'B' (0x42) at index 568

Answer 1

When using raw SQL and SQLAlchemy, you can use the sqlalchemy.text object to to ensure that parameter values are correctly quoted in SQL sent to the database ( docs ). The prevents both errors due to unquoted special characters or datatypes, as seen in this question, and also SQL injection attacks.

To use the text object, declare values in the SQL like this:

stmt = """INSERT INTO my_table (some_column) VALUES (:some_value)"""

or

stmt = """SELECT some_column FROM my_table WHERE some_column = :some_value"""

then create a text object

stmt = sqlalchemy.text(sql)

then use the bindparams method to bind the parameter value into the query

stmt = stmt.bindparams(some_value='whatever')

the resulting object can now be executed by the connection.

Here's an example based on your code:

import sqlalchemy as sa   

url = 'https://images-na.ssl-images-amazon.com/images/I/81Sp%2BP3kX-L._AC_SL1500_.jpg'
stmt = sa.text("""INSERT INTO links (url) VALUES (:url)""")
stmt = stmt.bindparams(url=url)

engine = sa.create_engine('mysql+pymysql://root:@localhost/test')

with engine.connect() as conn:
    conn.execute(stmt)

Answer 2

I think your error is caused by the combination of two factors:

1. you're constructing the query by directly pasting arguments into it – see comment by @rdas
2. your URL is url-encoded, ie. it contains a character encoded as %2B

You end up with a query containing the %2B sequence, which gets misinterpreted. Try creating a query with placeholders and passing arguments to conn.execute .