Filtering nans on condition in pandas dataframe

Question

I have a dataframe like this:

   COL1  COL2
0     1     2
1     1     NaN
2     2     6
3     2     8
4     3     10
5     3     NaN
6.    4     NaN

As you can see there are duplicates values in Col1, I want a dataframe that is like this:

   COL1  COL2
0     1     2
1     2     6
2     2     8
3     3     10
4     4     NaN

Basically if the same value in col1 has a nan value in col2 then I want to remove the row that has the NaN value. However, I am not trying to move NaN values all together if that is the only value available in Col2 for a value in Col1

Therefore, I know I cannot use this:

new_table = old_table[~old_table['COL2'].isna())

since this would remove all NaN values which is not necessarily what I am looking for

Answer 1

You can use pandas.DataFrame.duplicated with pandas.DataFrame.isna :

>>> df[~(df.COL1.duplicated(keep=False) & df.COL2.isna())]

   COL1  COL2
0     1   2.0
2     2   6.0
3     2   8.0
4     3  10.0
6     4   NaN