SimpleImputer ValueError

Question

I'm trying to replace NaNs with mean values, but when i do the fit method I get this error:

ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

This is my code:

from sklearn.impute import SimpleImputer
imp = SimpleImputer(missing_values = "NaN", strategy = "mean")
imp = imp.fit(df_missing.iloc[:, 1:4])

Before you flag it as repost, I know I might import numpy as well and use np.nan instead of "NaN", but isn't there a way to work around it and stick to pandas and SimpleImputer? I was following an online course and I'd like to stick to it as much as I can; unfortunately it's apparently outdated and, for example, it still runs

from sklearn.preprocessing import Imputer

Answer 1

I'm not sure if I understood your question clearly or not, but I assume instead of pd.NA or np.nan in the columns you want to impute, you have "NaN" (an object type). If that's the case, there's a way to work around it:

df_missing.iloc[:, 1:4] = df_missing.iloc[:, 1:4].replace({"NaN": np.nan})

By doing so, first you replace "NaN" with np.nan , and then you can use SimpleImputer:

imp = SimpleImputer(strategy = "mean")
imp = imp.fit(df_missing.iloc[:, 1:4])

I hope I understood your question right, and this solution helps you!

Answer 2

You can set imp object;

imp = SimpleImputer(missing_values = np.nan, strategy = "mean")