how to replace a NaN value in a column in data frame in pandas?

Question

i'm a newbie to pandas library and trying to do some analysis on the Titanic survival data using pandas. in the data set there are some cells in the Age column that doesn't have value so they appear in the data frame as 'NaN'. so i'm trying to replace these cells with the average of the male-ages for males and the average of the female ages for females. first i tried this

df[ df.Sex == "male"]['Age'].fillna( df[ df.Sex == "male" ]['Age'].mean())

well it works but doesn't change the values of the cells in the data frame when i print it later so i tried this

df[ df.Sex == "male"]['Age'].fillna( df[ df.Sex == "male" ]['Age'].mean(),inplace = True)

and got this error message

/anaconda3/lib/python3.4/site-packages/pandas/core/generic.py:2602: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame See the the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy self._update_inplace(new_data)

so anyone knows how to do it efficiently ?

Answer 1

what you are doing is right. you just have to reassign it to df:

df['Age'] = df['Age'].replace(df[(df['Sex']=='male') & (df['Age'].isnull())]['Age'], df['Age'].mean)

Answer 2

You should save the returned value from fillna (without inplace parameter) in a temporary variable, then loop through the indexes in that and change each of that index in df

tempdf = df[ df.Sex == "male"]['Age'].fillna( df[ df.Sex == "male" ]['Age'].mean())
for i in  tempdf.index:
    df['Age'][i] = tempdf[i]

Answer 3

fillna() accepts a dictionary with column names and values to replace NaN with. NaN in each column will be replaced with the coresponding value. Notice that other columns and the original df dataframe will not be affected if you don't specify inplace=True .

replacement_value = df[ df.Sex == "male" ]['Age'].mean()
new_df = df.fillna({'Age': replacement_value})

You can of course overwrite the existing dataframe:

df = df.fillna({'Age': replacement_value})
# or just df.fillna({'Age': replacement_value}, inplace=True)

Answer 4

尝试使用这个

df.loc[ df.Sex == "male",'Age'].fillna( df[ df.Sex == "male" ]['Age'].mean(),inplace = True)

Answer 5

Try this one

# Fill the nan values with the mean age
df['Age'] = df.groupby('Sex')['Age'].transform(lambda grp: grp.fillna(np.mean(grp)))

df

Got it from here