Pandas 将 function 应用于列表中的多个列

Question

I'm attempting to create a function that creates a modified dataframe with all outliers removed from the columns stored in my variable num_vars. Here is my function so far:

def remove_outliers(column):
    Q1 = np.percentile(df[column], 25, interpolation = 'midpoint')
    Q3 = np.percentile(df[column], 75, interpolation = 'midpoint')        
    IQR = Q3 - Q1
    IQR_mult = IQR * 1.5
    ceiling = Q3 + IQR_mult
    floor = Q1 - IQR_mult
    return df[(df[column] <= ceiling) & (df[column] >= floor)]

The columns I want to apply this function to are stored in

 num_vars = ['host_response_rate', 'accommodates', 'bedrooms', 'beds', 'minimum_nights', 'availability_30', 'number_of_reviews', 'review_scores_rating', 'review_scores_cleanliness', 'review_scores_checkin', 'review_scores_communication', 'review_scores_location', 'review_scores_value', 'time_from_last_review', 'num_amenities', 'price']

The function works when only calling one column such as 'price', but does not return a clean dataframe when calling multiple items at once. How can I make it so the function can take all these columns at once, and return a dataframe where all outliers have been removed?

Answer 1

You can change your remove_outlier function to accept row and column arguments, then iterate through the columns with the function applied to the rows as below:

def remove_outliers(row,column):
    Q1 = np.percentile(df[column], 25, interpolation = 'midpoint')
    Q3 = np.percentile(df[column], 75, interpolation = 'midpoint')
    IQR = Q3 - Q1
    IQR_mult = IQR * 1.5
    ceiling = Q3 + IQR_mult
    floor = Q1 - IQR_mult
    if float(ceiling)>= row[column] >= int(floor):
        return row[column]
    else:
        return None


num_vars = ['host_response_rate', 'accommodates', 'bedrooms', 'beds', 'minimum_nights', 'availability_30',
            'number_of_reviews', 'review_scores_rating', 'review_scores_cleanliness', 'review_scores_checkin',
            'review_scores_communication', 'review_scores_location', 'review_scores_value', 'time_from_last_review',
            'num_amenities', 'price']
for col in num_vars[:3]:
    df[col] = df.apply(lambda row: remove_outliers(row,col), axis=1)
df = df.dropna().reset_index(drop=True)