Data insertion not reflecting in mysql database

Question

I was trying this post: How to speed up Python CSV Read to MySQL Write to speed up my data insertion script. my table format: '''CREATE TABLE pattern ( pattern_id CHAR(32) PRIMARY KEY, block CHAR(4), pattern_source ENUM('LABEL','BURST'), pattern VARCHAR(150), pattern_type ENUM('PROD','EVAL','RMA','QMFAT','BURN_IN','DEBUG','UNKNOWN','REPAIR','VALIDATION') )''' I modified based on my database config. but after running the script, it runs without errors and data not updating in table. please refer to tick mark code snippet in that post above. find my code here.

 import mysql.connector import time import concurrent.futures import csv import itertools CSVFILE='C:\\Users\\Downloads\\pattern.csv' CHUNK=10_000 def doBulkInsert(rows): with mysql.connector.connect(user='root', password='zxcasdQWE123', host='localhost', database='testdatabase') as connection: connection.cursor().executemany(f'INSERT INTO table_name (pattern_id, block, pattern_source, pattern, pattern_type) VALUES (%s, %s, %s, %s, %s)',rows) connection.commit() def main(): _s = time.perf_counter() with open(CSVFILE) as csvfile: csvdata = csv.reader(csvfile) _s = time.perf_counter() with concurrent.futures.ThreadPoolExecutor() as executor: while (data:= list(itertools.islice(csvdata, CHUNK))): executor.submit(doBulkInsert, data) executor.shutdown(wait=True) print(f'Duration = {time.perf_counter()-_s}') if __name__ == '__main__': main()

can you tell or an updated code why it is not updating.

Answer 1

One thing you should always do with strings, is to use '' around their names. When there is a space in the name, the query will fail. So instead of:

VALUES (%s, %s, %s, %s, %s)

You could try:

VALUES ('%s', '%s', %s, '%s', %s)

Next issue, the pattern_id should be unique for each and every insert, else mysql will complain and only keep the first. You must be absolutely sure that pattern_id's are unique, If not, create a new column for the key. even when you never use that number.

Also, can you print the Sql statements before you run them. I'm not sure whether your Python code is correct.