Insert pandas DataFrame() into an MySQL table raises ProgrammingError

Question

Given a dataframe ( df ) in which all rows of 1 column contains a list basically the respective column can be seen as a list of lists. The rest of the columns are str , int or bool . The first row ( df.iloc[0] ) print the below:

name                                  John
isMale                                True
age                                     30
hobbies  ['hiking', 'gaming', 'reading', …
Name: 0, dtype: object

This is my code:

engine = create_engine('mysql+mysqlconnector://{user}:{password}@{host}:{port}/{database}'.format(**config))
df.to_sql(name='test', con=engine, if_exists='append', index=True)

The above executes:

[SQL: INSERT INTO test(`name`, `isMale`, age, hobbies) VALUES (%(name)s, %(isMale)s, %(age)s, %(hobbies)s)]
[parameters: {'index': 0, 'name': 'John', 'isMale': 1, 'age': 50, 'hobbies': ['hiking', 'gaming', 'reading', ... (84 characters truncated) ...  ]}]

I want to insert the df into a mysql table but sqlalchemy raises the following exception:

Traceback (most recent call last):
        return getattr(self, "_{0}_to_mysql".format(type_name))(value)
    AttributeError: 'MySQLConverter' object has no attribute '_list_to_mysql'

The above exception was the direct cause of the following exception:
    sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) Failed processing pyformat-parameters; Python 'list' cannot be converted to a MySQL type

---

• Is there a way to insert a python list as a cell value?
• How can I convert a python list to a valid MySQL type?

---

NB :

• The column hobbies is formatted as text utf8_general_ci

References :

• Similar error: https://bugs.mysql.com/bug.php?id=89112
• Background on this error at: http://sqlalche.me/e/13/f405

Answer 1

Try converting it all to regular strings. Just apply a lambda function on your name column. For example:

df = pd.DataFrame({'a':[1,2,3,4], 'b':[['a', 'b','v'], ['s', 'e','r'], ['a', 'j','k'], ['f','g','d']]})

df.b.apply(lambda x: ", ".join(x))
Out[31]: 
0    a, b, v
1    s, e, r
2    a, j, k
3    f, g, d
Name: b, dtype: object

Strings should definitely work fine with sql.