pandas to_dict with python native datetime type and not timestamp

Question

I have a pandas DataFrame df that contains Timesatamp columns.

I wish to create an iterator of rows (either via the iter.. methods or via to_dict ) from df where the Timesatamp values are python datetime .

I have tried doing this

for col in df.select_dtypes(['datetime']):
        df[col] = df[col].dt.to_pydatetime()

however it seems like the columns is still Timesatamp when using the above mentioned iterator methods. Is there a 'batch'y way to achieve this apart from manualy converting each values when its iterated upon?

---

example

df = pd.DataFrame({'d': pd.date_range('2018-01-01', freq='12h', periods=2), 'a':[1,2]})
for col in df.select_dtypes(['datetime']):
    df[col] = df[col].dt.to_pydatetime()
print(df.to_dict('records'))

the output:

[{'d': Timestamp('2018-01-01 00:00:00'), 'a': 1}, {'d': Timestamp('2018-01-01 12:00:00'), 'a': 2}]

the desired output:

[{'d': datetime.datetime(2018, 1, 1, 0, 0), 'a': 1}, {'d': datetime.datetime(2018, 1, 1, 12, 0), 'a': 2}]

Answer 1

You can try

df[col] = pd.Series(df[col].dt.to_pydatetime(), dtype = object)

instead of

df[col] = df[col].dt.to_pydatetime()

Answer 2

One workaround is the following:

#Initialize empty records list
records=[]

#Iterate over datetime columns
for col in df.select_dtypes(['datetime']):

    #Create a temp list consisting of dictionaries
    temp_df=[{col: r.to_pydatetime()} for r in df[col]]

    #Add it to records
    records+=temp_df

which result is:

records

[{'d': datetime.datetime(2018, 1, 1, 0, 0)},
 {'d': datetime.datetime(2018, 1, 2, 0, 0)}]

Answer 3

Try it:

df["d"]=df.d.apply(lambda t: t.date())                                                                              
df.d.to_dict()                                                                                                      

{0: datetime.date(2018, 1, 1), 1: datetime.date(2018, 1, 2)}