MySQL Stored Procedure DELETE not working when called from PYTHON

Question

I have a MySQL stored proc that will back up database records from a beyond a specified INTERVAL to a csv file, then DELETE the same records.

When I call the procedure from phpMyAdmin or MySQL Workbench it works perfect. It creates the archive file, and deletes the records.

However, when I call the stored procedure from within my Python application, it will create the archive file, but does not delete the records.

The input and session variables are also output to specific log files for debugging. This all works fine and I see what I expect in the log files.

Here is my stored proc:

CREATE DEFINER=`ctuser`@`%` PROCEDURE `sp_TableDump`(
    IN db VARCHAR(50), 
    IN tbl VARCHAR(50),
    IN dateCol VARCHAR(20),
    IN intrvl VARCHAR(20),
    IN allowArchive boolean,
    IN allowPurge boolean
)
BEGIN
    #I know my input fields are correct because of the log1.log created below
    SELECT db, tbl, dateCol, intrvl, allowArchive, allowPurge into outfile '/var/lib/mysql-files/log1.log';

    SET @defMaxLength = (SELECT @@group_concat_max_len);#get the default group_concat limit    
    SET SESSION group_concat_max_len = 1000000;#set the Group Concat Max Limit for tables with a lot of columns

    SET @date = CURDATE();
    SET @path = CONCAT('/var/lib/mysql-files/', db, '.', tbl, '_ARCHIVE_', @date, '.csv');

    SET @tbl = CONCAT(db, '.', tbl);  

    SET @where = CONCAT(' WHERE ', dateCol, ' < CURDATE() - INTERVAL ', intrvl);
    SET @orderBy = CONCAT(' ORDER BY ', dateCol); 

    #I know my session variables are correct because of the log2.log created below
    SELECT @date, @path, @tbl, @where, @orderBy into outfile '/var/lib/mysql-files/log2.log';

    IF allowArchive THEN    
        #I know we get in here because log2.log gets created            
        #archive the records
        #get the columns from the table
        SET @cols = (SELECT GROUP_CONCAT(QUOTE(`column_name`)) AS columns FROM information_schema.columns WHERE table_schema = db AND table_name = tbl);
        SET @colQry = CONCAT('(SELECT ', @cols, ')');      
        SET @outfileQry = CONCAT('(SELECT * FROM ', @tbl, @where, @orderBy, ' INTO OUTFILE \'', @path, 
                                '\' FIELDS ENCLOSED BY \'\\\'\' TERMINATED BY \'\t\' ESCAPED BY \'\' LINES TERMINATED BY \'\n\')');

        SET @outfileSQL = CONCAT(@colQry, ' UNION ALL ', @outfileQry);
        SELECT @cols, @colQry, @outfileQry, @outfileSQL into outfile '/var/lib/mysql-files/log2.log';

        PREPARE outStmt FROM @outfileSQL;
        EXECUTE outStmt;#This works every time 
        DEALLOCATE PREPARE outStmt;
    END IF;

    IF allowPurge THEN    
        #I know we get in here because log3.log gets created
        #delete the records
        SET @delSQL = CONCAT('DELETE FROM ', @tbl, @where);

        SELECT @delSQL into outfile '/var/lib/mysql-files/log3.log'; # @delSQL looks correct in log3.log

        PREPARE delStmt FROM @delSQL;        
        EXECUTE delStmt; # This does not seem to happen when I run in Python.  It works perfectly in MySQL Workbench or phpMyAdmin
        DEALLOCATE PREPARE delStmt;
    END IF;

    SET SESSION group_concat_max_len = @defMaxLength;#reset the default group_concat limit
END

Here is the Python function:

def archiveAndDeleteRecords():
    log.info('Attempting to purge data older than ' + interval + ' from ' + db + '.' + tbl + " col: " + columnName)    
    dateTime = datetime.now().strftime("%Y%m%d%H%M%S")
    archivePath = "/mnt/Storage/DatabasePurge/" + db + "/"

    try:
        args = [db, tbl, columnName, interval, allowArchive, allowPurge] # The args look good in python and when passed to stored proc
        stProc = 'sp_TEST'
        log.info('calling stproc ' + stProc)
        log.info('args ' + str(args))
        result_args = cursor.callproc(stProc, args)
        conn.commit()
    except Exception as ex:
        log.error("Could not execute stored procedure!");
        log.error("error: " + str(sys.exc_info()[0]));
        log.error("exception: " + str(ex));
        traceback.print_tb(ex.__traceback__)

    return;

I have been pounding my head against the wall for hours with this. Any help would be appreciated. Thanks.

Answer 1

This was also a headache for me, but I found a solution. On the one hand, your stored procedure has to work in your database. On the other hand, in python you will use sqlalchemy to connect to the corresponding database. NOTE: update the variable for your corresponding values.

from sqlalchemy import create_engine

driver='ODBC Driver 17 for SQL Server'
engine = create_engine('mssql+pyodbc://{}:{}@{}/{}?driver={}'.format(user_name, password, server_name, database_name, driver))

query = ('exec name_stored_procedure @var1=\'{}\', @var2=\'{}\''.format(value1, value2))

with engine.begin() as conn:
    conn.execute(query)

Using the begin method will execute the stored procedure and commit it to the database, deleting the data you desire