Psycopg2 “Can't execute an empty query”

Question

I saved some queries in a single .sql script, instead of writing them directly inside variables in a .py script.

Now I want to read this sql script using Python, split each sql statement by semicolon and then pass each of these scripts to Psycopg2 cursor for sequential execution.

The python script seems to read the file correctly, but a "can't execute an empty query" error is thrown when I try to execute the statements.

Maybe the catch is: there are many line-breaks spread across this sql script. Statements in it are written like below:

DROP TABLE IF EXISTS
    target_schema.some_table
;

SELECT
    column
FROM
    schema.table
;

Here's the python code:

import psycopg2
import pathlib

conn = psycopg2.connect(
     user=user
    ,password=password
    ,host=host
    ,port=port
    ,database=database
)

pg_cursor = conn.cursor()

scriptContents = Path('my_folder\my_sql_script.sql').read_text(encoding='utf-8')

sqlStatements = scriptContents.split(sep=';')

for statement in sqlStatements:    

    try:
        pg_cursor.execute(f'{statement}')
        conn.commit()
    except psycopg2.Error as errorMsg:
        print(errorMsg)        
        conn.rollback()

Could anyone help me solve this?

Answer 1

When you split your sql file by ; , the list created will have empty string as the last element and it will fail to execute as a valid query. In order to avoid it, giving that all the sqls in your files have ; in the end, try the following:

for statement in sqlStatements[:-1]:
# it will slice out the last element of your sqlStatements list
    try:
        pg_cursor.execute(f'{statement}')
        conn.commit()
    except psycopg2.Error as errorMsg:
        print(errorMsg)        
        conn.rollback()

Another solution is write your statements new line separated and then read them one by one in the following way:

my_sql_script.sql:

DROP TABLE IF EXISTS target_schema.some_table
SELECT column FROM schema.table

in order to run the statements, use:

with open('my_folder\my_sql_script.sql','r', encoding='utf-8') as f:
    for statement in f.readlines():    
        try:
            pg_cursor.execute(f'{statement.rstrip()}')
            conn.commit()
        except psycopg2.Error as errorMsg:
            print(errorMsg)        
            conn.rollback()