Pandas DataFrame removing NaN rows based on condition?

Question

Pandas DataFrame removing NaN rows based on condition.

I'm trying to remove the rows whose gender==male and status == NaN .

Sample df:

        name     status        gender   leaves
0       tom        NaN          male      5 
1       tom        True         male      6
2       tom        True         male      7
3       mary       True         female    1
4       mary       NaN          female    10
5       mary       True         female    15
6       john       NaN          male       2
7       mark      True          male       3

Expected Ouput:

        name     status        gender   leaves
0       tom        True         male      6
1       tom        True         male      7
2       mary       True         female    1
3       mary       NaN          female    10
4       mary       True         female    15
5       mark      True          male       3

Answer 1

You can use isna (or isnull ) function to get the rows with a value of NaN . With this knowledge, you can filter your dataframe using something like:

conditions = (df.gender == 'male')&(df.status.isna())
filtered_df = df[~conditions]

Answer 2

Good One given by @Derlin, other way I tried is using fillna() filled NaN with -1 and indexed them, just like below:

>>> df[~((df.fillna(-1)['status']==-1)&(df['gender']=='male'))]

Just for reference ~ operator is the same as np.logical_not() of numpy . So if you use this: df[np.logical_not((df.fillna(-1)['status']==-1)&(df['gender']=='male'))] (dont forget to import numpy as np ), means the same.