sklearn 估算行满足条件

Question

I'm trying to use sklearn SimpleImputer to impute missing ages from a particular column in a pandas DataFrame containing Titanic data. However, I want to separately impute the missing values for passengers whose names contain the word "Master" using the average of the other Master's ages.

I tried locating that data, and treating it separately:

imputer = SimpleImputer(strategy="mean")

# Copy data
imputed_X = X.copy()

# Get data for "masters"
masters = imputed_X[imputed_X['Name'].str.contains("Master")]

# Get imputed version of Age column
masters_age_imputed = pd.DataFrame(imputer.fit_transform(masters[["Age"]]))
masters_age_imputed.index = masters.index
# (So far so good... the missing values have been replaced with the average)

# But putting those values back into the DataFrame doesn't work:
imputed_X.loc[X['Name'].str.contains("Master"),"Age"] = masters_age_imputed

Instead of imputing all of the missing Master's ages with the average age, this deletes all of the non-missing ages, and replaces them with NaN.

Is there a better way of doing this? Eg, one that works? Aside from setting up my own for loop and replacing everything manually?

Answer 1

You need to fit first and use that to transform the data. When you fit the imputer you use the column along with the missing values. Use that fitted model to impute the missing values using transform as i used below.

Can you try this?

imp = Imputer(missing_values='NaN', strategy='mean', axis=0)
imp = imp.fit(imputed_X[['Age']])
imputed_X['Age'] = imp.transform(imputed_X[['Age']]).ravel()

If you want to only impute a subset of the data, (column name contains master for eg) then you can do that and merge it back to the original dataframe. You do not need looping, you can pd.merge it back.