dynamic insert using mysql and csv file

Question

I am learning python....I am trying to load a csv file into mysql. I am getting the below error. can someone please help?

   ordinal code:
   import csv
   import sqlalchemy




  NULL_FIELD_VALUE = r'\N'
  file_name = "C:/105AML11232020.txt"
  tbl_name="aml_master"

  engine = sqlalchemy.create_engine('mysql+mysqlconnector://root:xxxxx@localhost:3306/test',
                              connect_args={'auth_plugin': 'mysql_native_password'})
  connection = engine.connect()
  with open(file_name) as csvfile: 
      reader = csv.DictReader(csvfile, delimiter='|')

      insert_table = table(tbl_name,*[column(field) for field in reader.fieldnames])
      insert_dict = [{k: None if v == NULL_FIELD_VALUE else v for k,v in row.items()} for row in reader]
      connection.execute(insert_table.insert(), insert_dict)

---

Error:

 insert_dict = [{k: None if v == NULL_FIELD_VALUE else v for k,v in row.items()} for row in reader] ^ 
 IndentationError: unindent does not match any outer indentation level

---

Answer 1

Pandas really makes this kinds of things easy.

Here is an example:

import pandas as pd
import sqlalchemy

file_name = "C:/105AML11232020.txt"
tbl_name = "aml_master"

engine = sqlalchemy.create_engine(
    "mysql+mysqlconnector://root:xxxxx@localhost:3306/test",
    connect_args={"auth_plugin": "mysql_native_password"},
)

#Loads the file into a dataframe
data = pd.read_csv(file_name)

# Save the datafile to a table
data.to_sql(tbl_name, con=engine)

Check the pandas docs for DataFrame.read_csv and DataFrame.to_sql

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_sql.html