How do you run a complex sql script within a python program?

Question

I have a large SQL script that creates my database (multiple tables, triggers, and such. Using MySQL), and I need to execute that script from within a python program. My old code did this:

sql_file = open(os.path.join(self.path_to_sql, filename), 'r')
sql_text = sql_file.read()
sql_stmts = sql_text.split(';')
for s in sql_stmts:
    cursor.execute(s)

This worked fine until I started including triggers in my sql script. Since I now need to change the delimiter from ; to something else, in order to support triggers with multiple SQL statements in each, my code to split each statement into it's own string no longer works because the "delimiter |" line in my script fails to split properly, and I get a syntax error from cursor.execute(s).

So, I need some way to tell mysqldb to execute an entire sql script at once, instead of individual sql statements. I tried this:

sql_file = open(os.path.join(self.path_to_sql, filename), 'r')
sql_text = sql_file.read()
cursor.execute(sql_text)

However, I get the following error when I try to run that code: ProgrammingError: (2014, "Commands out of sync; you can't run this command now") My Google-fu tells me that this is because the Python mysqldb package doesn't support complex SQL statements being sent to cursor.execute().

So how can I do this? I'd really like to find a way to do this entirely within Python, so that the code will remain entirely portable. We have several programmers working on this project in Eclipse, some on Windows and some on Mac, and the code needs to work on the Linux production server as well.

If I can't use Python code to actually run the SQL, how can I tell Python to launch a separate program to execute it?

Answer 1

(not a python solution) you can use

  os.system('mysql < etc')

ooh, edit (python solution):

if you have the query broken up by line, you can close and reopen the cursor and execute by line.

redit: sorry, only skimmed your first paragraph. seems like you were doing this sort of thing at first.

Answer 2

This doesn't seem like a very good way to structure a multi-language program.

Brandon's answer is really the right way to go if all you're doing is just executing a big chunk of sql.

On the other hand, if you're doing stuff with the results of queries throughout the process of the job, then you shouldn't try to parse a large, wellformed sql script. Instead you should mix the sql statements into your python code.