Python — Value Error: unsupported format character ',' (0x2c) at index 968

Question

There are many similar questions on Stackoverflow and I have been digging for a while here. However, I still can't find a solution to the specific problem I have.

I am using Python Snowflake connector to extract data using sql files. My file directory looks like this:

SQL
   pay_gbd.sql
data_extract.py

pay_gbd.sql looks like the following, with two parameters -- start_pickup_data and end_pickup_date:

WITH
payment as
(
    select
          confirmation_number
        , payment_statuses
        , case
            when lower(payment_statuses) like '%%show%%' then 'NOSHOW'
            when lower(payment_statuses) like '%%cancel%%' then 'CANCEL'
            else 'SHOW'
          end as payment_status
    from payment_table
),

rawData as 
(   select *
    from booking_table g
    where
        g.pu_timestamp >= %(start_pickup_date)s and g.pu_timestamp < %(end_pickup_date)s
        and g.supplier_car_days > 0)

select *
from rawData;

As you can see I used '%%s%%' in like to avoid errors.

data_extract.py has the following code:

def executeSQLScriptsFromFile(filepath, param_dict):

    ctx = snowflake.connector.connect(
        user='USER_NAME',
        account='SECRET_1',
        region='us-east-1',
        warehouse='SECRET_2',
        database='SECRET_3',
        role='SECRET_4',
        password='SECRET_5')

    fd = open(filepath, 'r')
    query = fd.read()
    fd.close()
    print(query)
    cs = ctx.cursor()
    try:
        cur = cs.execute(query, param_dict)
        df = pd.DataFrame.from_records(iter(cur), columns=[x[0] for x in cur.description])
    finally:
        cs.close()
    ctx.close()

    return df

def extract_pay_gbd(start_pickup_date, end_pickup_date):
    pay_gbd_sqlpath = os.path.join(os.getcwd(), 'sql/pay_gbd.sql')

    print('Start extracting pay_gbd data from Snowflake, pickup date range: {} to {}'.format(start_pickup_date,                                                                                            end_pickup_date))
    param_dict = {'start_pickup_date': start_pickup_date, 'end_pickup_date': end_pickup_date}
    pay_gbd = executeSQLScriptsFromFile(pay_gbd_sqlpath, param_dict)

    return pay_gbd

However, when I run the extract_pay_gbd function, I always get the following error:

  File "C:\Users\...\data_extract.py", line 45, in executeSQLScriptsFromFile
    cur = cs.execute(query, param_dict)
  File "C:\Users\...\snowflake\connector\cursor.py", line 458, in execute
    query = command % processed_params
ValueError: unsupported format character ',' (0x2c) at index 968

Output for print(query) looks exactly the same as the query in the .sql file:

WITH
payment as
(
    select
          confirmation_number
        , payment_statuses
        , case 
            when lower(payment_statuses) like '%%show%%' then 'NOSHOW'
            when lower(payment_statuses) like '%%cancel%%' then 'CANCEL'
            else 'SHOW'
          end as payment_status
    from payment_table
),

rawData as 
(   select *
    from booking_table g
    where
        g.pu_timestamp >= %(start_pickup_date)s and g.pu_timestamp < %(end_pickup_date)s
        and g.supplier_car_days > 0)

select *
from rawData;

Any suggestions are highly appreciated!

Answer 1

Following hints by @JohnGordon, I found and solved the problem. Hope the solution here to be helpful to those new to Snowflake Python Connector.

The problem is I have a '%' in the comment part in the sql file. Removing it and the code will run perfect.