我如何从异常值中清除数据集，因为它包含Python中的数字和分类变量？

Question

I would like to clean my dataset from outliers but just in three specific columns, as the other 10 contain categorical variables. So how can I get my data cleaned by only referring to these specific columns?

I'd like to use iqr range method. That's the code i run so far:

import numpy as np
def outliers(x): 
       return np.abs(x- x.median()) > 1.5*(x.quantile(.75)-x.quantile(0.25))
ath2.Age[outliers(ath2.Age)]
ath2.Height[outliers(ath2.Height)]
ath2.Weight[outliers(ath2.Weight)]

After checking the number of outliers in the columns I'm interested in, I don't know how to proceed further.

Answer 1

If you want the code to be dynamic, you can 1st check the columns which are not categorical by below code:

cols = df.columns
num_cols = df._get_numeric_data().columns 
##num_cols will contains list of column names which are numeric
## In your case, it should come Age,Height etc.

Alternatively, you can also use include or exclude parameters using df.select_dtypes according to your dataframe

After this run below code from columns from above:

df[np.abs(df.Data-df.Data.mean()) <= (3*df.Data.std())]  
## Df is the dataframe and Data is the name of the column. 
#In your case, it will be Age,Height etc.

OR

If you want to make a new df with only the numerical columns and find out the outliers in one shot, below is the code:

df[df.apply(lambda x: np.abs(x - x.mean()) / x.std() < 3).all(axis=1)]