pandas set_index 和 reset_index 改变变量的类型

Question

Here is my pandas Dataframe

import datetime as dt
d1=dt.date(2021,1,1)
d2=dt.date(2021,1,13)
df = pd.DataFrame({'date':[d1,d2],'location':['Paris','New York'],'Number':[2,3]})

Here is my problem

df = df.set_index(['date'])
df = df.reset_index()
print(df.loc[df.date == d1]) # Ok !
df = df.set_index(['location','date'])  # order has not importance
df = df.reset_index()
print(df.loc[df.date == d1]) # Not OK ! it returns an Empty DataFrame 

It seems that when I set_index with two columns and reset_index the type of date is lost.

I don't understand why it is working in the first case and not in the second, where I need to do pd.to_datetime(df['date']).dt.date to work again?

Answer 1

Since it is change the column type to datetime64[ns] and the output for single value is Timestamp we need add date convert

df.date[0]
Out[175]: Timestamp('2021-01-01 00:00:00')

df.loc[df.date.dt.date == d1]
Out[177]: 
  location       date  Number
0    Paris 2021-01-01       2

Answer 2

To explain the difference, we can inspect the data type of the elements and see the difference.

Applying the type function on the dataframe at different stages:

df.applymap(type)

You can see from the following that the data type at the last stage is different. Hence, no match at the last stage.

Output:

import datetime as dt
d1=dt.date(2021,1,1)
d2=dt.date(2021,1,13)
df = pd.DataFrame({'date':[d1,d2],'location':['Paris','New York'],'Number':[2,3]})

df.applymap(type)

                      date       location         Number
0  <class 'datetime.date'>  <class 'str'>  <class 'int'>      # <=== datetime.date
1  <class 'datetime.date'>  <class 'str'>  <class 'int'>      # <=== datetime.date

df = df.set_index(['date'])
df = df.reset_index()
print(df.loc[df.date == d1]) # Ok !

df.applymap(type)

                      date       location         Number
0  <class 'datetime.date'>  <class 'str'>  <class 'int'>      # <=== datetime.date
1  <class 'datetime.date'>  <class 'str'>  <class 'int'>      # <=== datetime.date


df = df.set_index(['location','date'])  # order has not importance
df = df.reset_index()
print(df.loc[df.date == d1]) # Not OK ! it returns an Empty DataFrame 

df.applymap(type)

        location                                                date         Number
0  <class 'str'>  <class 'pandas._libs.tslibs.timestamps.Timestamp'>  <class 'int'>         # <=== data type changed to pandas._libs.tslibs.timestamps.Timestamp 
1  <class 'str'>  <class 'pandas._libs.tslibs.timestamps.Timestamp'>  <class 'int'>         # <=== data type changed to pandas._libs.tslibs.timestamps.Timestamp