pandas: “Lost connection to MySQL server” “system error: 32 Broken pipe”

Question

I am getting the above error when trying to import a pandas dataframe:

import pandas as pd
from sqlalchemy import create_engine
engine = create_engine('mysql://username:password@localhost/dbname')
c = getsomedata()
fields = ['user_id', 'timestamp', 'text']
c1 = c[fields].reset_index()
c1.to_sql(name='comments', con=engine, if_exists='replace', index=False)

There are lots of questions with this MySql issue - but how to address with a pandas import?

Answer 1

The solution for me was very simple: Use the chunksize option:

c1.to_sql(name='comments', con=engine, chunksize=1000, if_exists='replace', index=False)
                                       ^^^^^^^^^^^^^^^

Probably related to this issue with overly large packets.