Pandas - 如果dataFrame的所有值都是NaN

Question

How to create an if statement that does the following:

 if all values in dataframe are nan:
     do something 
 else: 
     do something else

According to this post , one can check if all the values of a DataFrame are NaN. I know one cannot do:

if df.isnull().all():
    do something

It returns the following error:

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Answer 1

Need another all , because first all return Series and another scalar :

if df.isnull().all().all():
    do something

Sample:

df = pd.DataFrame(index=range(5), columns=list('abcde'))
print (df)
     a    b    c    d    e
0  NaN  NaN  NaN  NaN  NaN
1  NaN  NaN  NaN  NaN  NaN
2  NaN  NaN  NaN  NaN  NaN
3  NaN  NaN  NaN  NaN  NaN
4  NaN  NaN  NaN  NaN  NaN

print (df.isnull())
      a     b     c     d     e
0  True  True  True  True  True
1  True  True  True  True  True
2  True  True  True  True  True
3  True  True  True  True  True
4  True  True  True  True  True

print (df.isnull().all())
a    True
b    True
c    True
d    True
e    True
dtype: bool

print (df.isnull().all().all())
True

if df.isnull().all().all():
    print ('do something')

If need faster solution - numpy.isnan with numpy.all , but first convert all values to numpy array by values :

print (np.isnan(df.values).all())
True

Timings :

df = pd.DataFrame(np.full((1000,1000), np.nan))
print (df)

In [232]: %timeit (np.isnan(df.values).all())
1000 loops, best of 3: 1.23 ms per loop

In [233]: %timeit (df.isnull().all().all())
100 loops, best of 3: 10 ms per loop

In [234]: %timeit (df.isnull().values.all())
1000 loops, best of 3: 1.46 ms per loop

Answer 2

Faster improvement on jezrael's would be df.isnull().values.all()

In [156]: df.isnull().values.all()
Out[156]: True

---

Benchmarks

small

In [149]: df.shape
Out[149]: (5, 5)

In [150]: %timeit df.isnull().values.all()
10000 loops, best of 3: 112 µs per loop

In [151]: %timeit df.isnull().all().all()
1000 loops, best of 3: 271 µs per loop

large

In [153]: df.shape
Out[153]: (1000, 1000)

In [154]: %timeit df.isnull().values.all()
10 loops, best of 3: 26.6 ms per loop

In [155]: %timeit df.isnull().all().all()
10 loops, best of 3: 40.8 ms per loop