How do I get Data to 'commit' from Python to SQL Server?

Question

I have a localhost SQL Server running and am able to connect to it successfully. However, I am running into the problem of data not transfering over from temp csv files. Using Python import pyodbc for Server connection.

I've tried with Python Import pymssql but have had worse results so I've stuck with pyodbc. I've also tried closing the cursor each time or just at the end but not to any luck.

Here is a piece of code that I am using. Towards the bottom are two different csv styles. One is a temp in which is used to fill the SQL Server table. The other is for my personal use to make sure I am actually gathering information at the moment but, in the long term will be removed so only the temp csv is used.

@_retry(max_retry=1, timeout=1)
def blocked_outbound_utm_scada():
    # OTHER CODE EXISTS HERE!!!
    # GET Search Results and add to a temp CSV File then send to MS SQL Server
    service_search_results_str = '/services/search/jobs/%s/results?output_mode=csv&count=0' % sid
    search_results = (_service.request(_host + service_search_results_str, 'GET',
                                       headers={'Authorization': 'Splunk %s' % session_key},
                                       body={})[1]).decode('utf-8')
    with tempfile.NamedTemporaryFile(mode='w+t', suffix='.csv', delete=False) as temp_csv:
        temp_csv.writelines(search_results)
        temp_csv.close()
        try:
            cursor.execute("BULK INSERT Blocked_Outbound_UTM_Scada FROM '%s' WITH ("
                           "FIELDTERMINATOR='\t', ROWTERMINATOR='\n', FirstRow = 2);" % temp_csv.name)
            conn.commit()
        except pyodbc.ProgrammingError:
            cursor.execute("CREATE TABLE Blocked_Outbound_UTM_Scada ("
                           "Date_Time varchar(25),"
                           "Src_IP varchar(225),"
                           "Desktop_IP varchar(225));")
            conn.commit()
        finally:
            cursor.execute("BULK INSERT Blocked_Outbound_UTM_Scada FROM '%s' WITH ("
                           "FIELDTERMINATOR='\t', ROWTERMINATOR='\n', FirstRow = 2);" % temp_csv.name)
            conn.commit()
        os.remove(temp_csv.name)
    with open(_global_path + '/blocked_outbound_utm_scada.csv', 'a', newline='') as w:
        w.write(search_results)
    w.close()

I'm just trying to get the information into SQL Server but the code seems to be ignoring cursor.commit() . Any help is appreciated in figuring out what is wrong.

Thanks in Advance!

Answer 1

Try it without the conn.commit .

I do not understand why or how does it work but it seems, as well to me, that pyodbc ignores the commit clause.

Answer 2

Try change autocommit parameter in pymssql.connect()

conn = pymssql.connect(host=my_host, user=my_user, password=my_password, database=my_database, autocommit=True)
conn = pymssql.connect(host=my_host, user=my_user, password=my_password, database=my_database, autocommit=False)