change dataframe values into np.nan using apply returns 0.0

Question

This is my code where this_proteinDF.loc[:, 'p-value'] contains masked values:

this_proteinDF.loc[:, 'p-value'] = this_proteinDF.loc[:, 'p-value'].apply(lambda x: np.nan if x is np.ma.masked else x)

This should change every masked value into np.nan , however when I call the values after this operation it returns a float64 0.0 instead of nan .

What's going on here and how do i fix this?

Answer 1

It seems you have several points of mis-understanding (you and me).

1. np.ma.masked is a numpy constant that supposed to mean that something is masked. I do not think it means what you think it means.
2. I'm not sure when you actually want the 'p-value' series masked.

I'm thinking it's beneficial to you and maybe others to show the following.

mask method
use pandas to mask the things you'd like

df = pd.DataFrame({'p-value': [.001, .01, .1, .2, .05, .5]})
df

Check what's not significant

df['p-value'].gt(.05)

0    False
1    False
2     True
3     True
4    False
5     True
Name: p-value, dtype: bool

mask will make True values into np.nan values

df.loc[:, 'is_sig'] = df['p-value'].mask(df['p-value'].gt(.05))
df