在一行的 dataframe 中将字符串转换为日期时间

Question

I have a CSV file that serves as an input to my calculations. The file contains a table that is read as a dataframe in my script. Two columns are datetime objects. When I am reading the file to a dataframe I am converting those values from default string type to datatime applying to_datetime function with UTC argument. The code is below:

import pandas as pd

df_in = pd.read_csv('./Out/in.csv', index_col = 0, usecols = [0, 1, 2, 3, 4], header = 0)

df_in.iloc[:,2:4] = df_in.iloc[:,2:4].apply(pd.to_datetime, utc = True)

print(df_in)

Works like a charm on the following input CSV:

,Rig ID,Rig Name,Start Time,End Time
60,5,D004,2020-05-08 02:45:14.664341,2020-05-10 08:12:29.719839
70,5,D004,2020-05-14 07:59:09.280761,2020-05-15 02:11:52.358921

The moment the CSV file is reduced to 1 line, eg:

,Rig ID,Rig Name,Start Time,End Time
60,5,D004,2020-05-08 02:45:14.664341,2020-05-10 08:12:29.719839

I am getting the error

TypeError: data type not understood.

As far as I can understand it is something to do with slice assignment on single line dataframe

df_in.iloc[:,2:4] =

but I am struggling to find its cause and "smart" workaround.

Appreciate your help!

Answer 1

I've found that with older versions of pandas this can happen with when date fields are present in the data frame (specifically datetime64 variants). So, instead of iloc try this...

df_in[['Start Time','End Time']] = df_in[['Start Time','End Time']].apply(pd.to_datetime, utc = True)