pandas dataframe with Date index -> insert into MySQL

Question

The object df is of type pandas.core.frame.DataFrame.

In [1]: type(df)
Out[1]: pandas.core.frame.DataFrame

The index of df is a DatetimeIndex

In [2]: type(df.index)
Out[2]: pandas.tseries.index.DatetimeIndex

And con gives a working MySQLdb connection

In [3]: type(con)
Out[3]: MySQLdb.connections.Connection

I've not been able to get this dataframe entered into a MySQL database correctly, specifically, the date field comes through as null when using the following (as well as some variations on this).

df.to_sql( name='existing_table',con=con, if_exists='append', index=True, index_label='date', flavor='mysql', dtype={'date': datetime.date})

What are the steps required to have this dataframe entered correctly into a local MySQL database, with 'date' as a date field in the db?

Answer 1

To correctly write datetime data to SQL, you need at least pandas 0.15.0.

Starting from pandas 0.14, the sql functions are implemented using SQLAlchemy to deal with the database flavor specific differences. So to use to_sql , you need to provide it an SQLAlchemy engine instead of a plain MySQLdb connection:

from sqlalchemy import create_engine
engine = create_engine('mysql+mysqldb://....')

df.to_sql('existing_table', engine, if_exists='append', index=True, index_label='date')

Note: you don't need to provide the flavor keyword anymore.

Plain DBAPI connections are no longer supported for writing data to SQL, except for sqlite.

See http://pandas.pydata.org/pandas-docs/stable/io.html#io-sql for more details.