熊猫如何在所有浮点数均为NaN时删除行

Question

I have the following df

  AAA BBB CCC DDD  ID1  ID2  ID3  ID4
0 txt txt txt txt  10   NaN  12   NaN
1 txt txt txt txt  10   NaN  12   13
2 txt txt txt txt  NaN  NaN  NaN  NaN

With the following dtypes

AAA          object
BBB          object
CCC          object
DDD          object
ID1          float64
ID2          float64
ID3          float64
ID4          float64

Is there a way to drop rows only when ALL float columns are NaN?

output:

  AAA BBB CCC DDD  ID1  ID2  ID3  ID4
0 txt txt txt txt  10   NaN  12   NaN
1 txt txt txt txt  10   NaN  12   13

I can't do it with df.dropna(subset=['ID1','ID2','ID3','ID4']) because my real df has several dynamic floating columns.

Thanks

Answer 1

Use DataFrame.select_dtypes for get all float columns, then test for non missing values and select by DataFrame.any for at least one non misisng value per row - so misising floats rows are removed:

df1 = df[df.select_dtypes(float).notna().any(axis=1)]
print (df1)
   AAA  BBB  CCC  DDD   ID1  ID2   ID3   ID4
0  txt  txt  txt  txt  10.0  NaN  12.0   NaN
1  txt  txt  txt  txt  10.0  NaN  12.0  13.0

Your solution with DataFrame.dropna should be changed for pass float columns and parameter how='all' for test if all NaN s per rows:

df1 = df.dropna(subset=df.select_dtypes(float).columns, how='all')
#for return same dataframe 
#df.dropna(subset=df.select_dtypes(float).columns, how='all', inplace=True)

If possible multiple types of floats check by np.floating :

df1 = df.dropna(subset=df.select_dtypes(np.floating).columns, how='all')

Answer 2

Use

df.dropna(subset=df.select_dtypes(include=np.number).columns, how='all')

---

I'd suggest using include=np.number because it includes all float dtypes - which all may contain NaN . When you use include=float , you just get the standard npfloat64 dtype

For illustration:

df['ID5'] = np.array([1,2,np.nan], dtype=np.float16)


>>> df.select_dtypes(include=float).columns.tolist()
['ID1', 'ID2', 'ID3', 'ID4']

>>> df.select_dtypes(include=np.number).columns.tolist()
['ID1', 'ID2', 'ID3', 'ID4', 'ID5']

Answer 3

您可以将NaN替换为0 ，然后删除仅包含NaN列

df.loc[:,~df.replace(0,np.nan).isna().all()]