如何从数据框中删除具有空值和分类变量的行？

Question

I am trying to drop the rows with null values and categorical variables from the dataframe that I imported from Excel. I've tried many other functions and many different ways to do so as well but I am not able to drop them, at least not all.

There are around 185000 rows with 6 columns. What I was trying to do is using for loop to go through the entire rows and drop the rows if there is a null value or categorical variable on the column "Order ID".

This is one of the codes I've tried:

f = 0

value = merged_file.at[f, 'Order ID']
for value in merged_file:
    if value is None:
        merged_file.drop(merged_file.index[f])
        merged_file.reset_index(inplace=True, drop=True)
        f+=1
        continue
    elif value == 'Order ID':
        merged_file.drop(merged_file.index[f])
        merged_file.reset_index(drop=True, inplace=True)
        f+=1
        continue
    elif f==186845:
        break
    else:
        f+=1
        continue

I would be grateful if correct me what I am doing wrong and please let me know if there is a better way to specify and drop the rows or columns with null values and categorical variables.

Thank you.

Answer 1

So, it seems you're using pandas even if the code does not look really pythonic. Anyway, I would suggest to not iterate though each row of the dataframe, in pandas rows containing nan can be dropped using dropna :

 merged_file.dropna(subset=['Order ID'],inplace=True)

To remove the rows containing categorical variables instead you can use numpy isreal . Apply simply apply the function isreal to all rows, labelling as False all rows which do not contain numerical values.

import numpy as np
merged_file = merged_file[merged_file['Order ID'].apply(lambda x: np.isreal(x))]