Filter Pandas data frame based on criteria - fails on NaN values

Question

I'm trying to filter a Pandas data frame based on a criteria (Python 2.7):

data[data['a']=='bbb']

But some of the values in the series data['a'] are NaN and I get an error: invalid type comparison .

How can I ignore it and treat the NaN as not matching the criteria thus filtering it out?

Thanks!!

Answer 1

You can try notnull :

data[(data['a']=='bbb') & (data['a'].notnull())]

Sample:

print data
      a
0   bbb
1   bbb
2   bbb
3   bbb
4   bbb
5   bbb
6   bbb
7   NaN
8     a
9     a
10  bbb

print data[(data['a']=='bbb') & (data['a'].notnull())]
      a
0   bbb
1   bbb
2   bbb
3   bbb
4   bbb
5   bbb
6   bbb
10  bbb

Answer 2

Reassign the column type as follows:

df['a'] = df['a'].astype('O')

This should solve the issue.