熊猫数据框列是否可以具有datetime.date类型？

Question

I am using cx_oracle to fetch date from databases. I would like to put the fetched data into a pandas dataframe. My problem is that the dates are converted to numpy.datetime64 objects which I absolutely don't need.

I would like to have them as datetime.date objects. I have seen the dt.date method but it still gives back numpy datetypes.

Answer 1

Edit: It appears that with pandas 0.21.0 or newer, there is no problem holding python datetime.date s in a DataFrame. date-like columns are not automatically converted to datetime64[ns] dtype.

import numpy as np
import pandas as pd
import datetime as DT

print(pd.__version__)
# 0.21.0.dev+25.g50e95e0
dates = [DT.date(2017,1,1)+DT.timedelta(days=2*i) for i in range(3)]
df = pd.DataFrame({'dates': dates, 'foo': np.arange(len(dates))})
print(all([isinstance(item, DT.date) for item in df['dates']]))
# True
df['dates'] = (df['dates'] + pd.Timedelta(days=1))
print(all([isinstance(item, DT.date) for item in df['dates']]))
# True

---

For older versions of Pandas:

There is a way to prevent a Pandas DataFrame from automatically converting datelike values to datetime64[ns] by assigning an additional value such as an empty string which is not datelike to the column. After the DataFrame is formed, you can remove the non-datelike value:

import pandas as pd
import datetime as DT
dates = [DT.date(2017,1,1)+DT.timedelta(days=i) for i in range(10)]
df = pd.DataFrame({'dates':['']+dates})
df = df.iloc[1:]
print(all([isinstance(item, DT.date) for item in df['dates']]))
# True

Clearly, programming this kind of shenanigan into serious code feels entirely wrong since we're subverting the intent of the developers. There are also computational speed advantages to using datetime64[ns] s over lists or object arrays of datetime.dates . Moreover, if df[col] has dtype datetime64[ns] then df[col].dt.date.values returns an object NumPy array of python datetime.date s:

import pandas as pd
import datetime as DT
dates = [DT.datetime(2017,1,1)+DT.timedelta(days=2*i) for i in range(3)]
df = pd.DataFrame({'dates': dates})
print(repr(df['dates'].dt.date.values))
# array([datetime.date(2017, 1, 1), datetime.date(2017, 1, 3),
#        datetime.date(2017, 1, 5)], dtype=object)

So you could perhaps enjoy the best of both worlds by keeping the column as datetime64[ns] and using df[col].dt.date.values to obtain datetime.date s when necessary.

On the other hand, the datetime64[ns] s and Python datetime.date s have different ranges of representable dates.

• datetime64[ns] s can represent datetimes from 1678 AD to 2262 AD .
• datetime.date s can represent dates from DT.date(0,1,1) to DT.date(9999,1,1) .

If the reason why you want to use datetime.date s instead of datetime64[ns] s is to overcome the limited range of representable dates, then perhaps a better alternative is to use a pd.PeriodIndex :

import pandas as pd
import datetime as DT
dates = [DT.date(2017,1,1)+DT.timedelta(days=2*i) for i in range(10)]
df = pd.DataFrame({'dates':pd.PeriodIndex(dates, freq='D')})
print(df)
#        dates
# 0 2017-01-01
# 1 2017-01-03
# 2 2017-01-05
# 3 2017-01-07
# 4 2017-01-09
# 5 2017-01-11
# 6 2017-01-13
# 7 2017-01-15
# 8 2017-01-17
# 9 2017-01-19