Execute .sql files that are used to run in SQL Management Studio in python

Question

As part of artifacts delivery, our developers give the data and structure scripts in .sql files. I usually "double click" on these files to open in "Microsoft SQL Server Management Studio". Management studio will prompt me for entering database server and user/pwd. I enter them manually and click on Execute button to execute these scripts.

These scripts contain structure and data sql commands. Each script may contain more than one data command (like select, insert, update, etc). Structure and data scripts are provided in separate .sql files.

These scripts also contain stored procedures and functions, etc. They also contain comments / description.

I want to automate execution of these scripts through python. I looked at pyodbc and pymssql but they dont look like solve my issue. Through pyodbc, i need to read each .sql file and read the sql commands and execute them one by one. As the files may have comments / description / SPs / etc, reading the files will be little difficult.

Can anyone give suggestion on how to automate this?

Thanks in advance.

Answer 1

You could just run them using sqlcmd. Sqlcmd is a command line utility that will let you run .sql scripts from the command line, which I'm sure you can kick off through python.

Answer 2

If the file is not too big for memory, you can parse it using this code and run each statement using pymssql .

It will execute whenever it finds a GO line or a ; at the end of the line.

    _conn = pymssql.connect(** connection settings **)
    _cur = _conn.cursor()
    with open(filename, 'r') as f:
        script = f.read().decode('utf-8')  # or whatever its encoding is
    script = re.sub(r'\/\*.*?\*\/', '', script, flags=re.DOTALL)  # remove multiline comment
    script = re.sub(r'--.*$', '', script, flags=re.MULTILINE)  # remove single line comment

    sql = []
    do_execute = False
    for line in script.split(u'\n'):
        line = line.strip()
        if not line:
            continue
        elif line.upper() == u'GO':
            do_execute = True
        else:
            sql.append(line)
            do_execute = line.endswith(u';')

        if do_execute and filter(None, sql):  # ignore if only blank lines
            cursor.execute(u'\n'.join(sql).encode("cp1252"))  # I have experienced problems when executing utf-8
            do_execute = False
            sql = []

    _conn.close()