理解 to_datetime AttributeError: 'tuple' 对象没有属性 'lower'

Question

I have inherited some very messy legacy code that calls Google Analytics API and saves some metrics into a pandas dataframe.

There is a date column which is a string and I want to convert to a date. I would normally use pd.to_datetime, like this final['date'] = pd.to_datetime(final['date'], format='%Y%m%d', errors='coerce') but it gives me this error:

AttributeError: 'tuple' object has no attribute 'lower'

This is description of the dataframe:

print(final.info())

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 37047 entries, 0 to 37046
Data columns (total 5 columns):
(date,)               37047 non-null object
(landingPagePath,)    37047 non-null object
(sessions,)           37047 non-null object
(bounces,)            37047 non-null object
(market,)             37047 non-null object
dtypes: object(5)
memory usage: 1.4+ MB
None

I have found a similar SO question here and the solution works if I do

final['date'] = pd.to_datetime([x for x in final['date'].squeeze().tolist()], dayfirst=True) .

However I still do not understand what caused the issue in the first place. I guess that the column is somehow inconsistent, but I do not know how to identify which row is the culprit. What's the best way to find out?

Answer 1

as in my very similar case, please check type(final['date']) . pd.to_datetime() works mostly with pd.Series .

This could even explain why squeeze() works. It converts DataFrame to Series .