[英]How to remove a outliers with z-scores (3 or -3) using apply function
[英]How to find the outliers from the data set and plot using Z score
數據集如下
store id,revenue ,profit
101,779183,281257
101,144829,838451
101,766465,757565
101,353297,261071
101,1615461,275760
102,246731,949229
102,951518,301016
102,444669,430583
代碼如下
import pandas as pd
dummies1 = dummies[['storeid', 'revenue', 'profit']]
cols = list(dummies1.columns)
cols.remove('storeid')
dummies1[cols]
# code to find the z score
for col in cols:
col_zscore = col + '_zscore'
dummies1[col_zscore] = (dummies1[col] - dummies1[col].mean())/dummies1[col].std(ddof=0)
在這里我需要散點圖,框 plot 與異常值,怎么辦
如何找到異常值如下?
假設threshold is 3
意味着 np.abs(z_score) > 閾值將被視為異常值。
根據 z-score 對數據進行切片,您將獲得 plot 的數據。 如果您只想找到一個變量是異常值的位置,您可以執行以下操作(例如):
THRESHOLD = 1.5 #nothing > 3 in your example
to_plot = dummies1[(np.abs(dummies1['revenue_zscore']) > THRESHOLD)]
或者,如果任一列可能是異常值,您可以執行以下操作:
to_plot = dummies1[(np.abs(dummies1['revenue_zscore']) > THRESHOLD) |
(np.abs(dummies1['profit_zscore']) > THRESHOLD)]
您對 plot 不是很具體,但這是一個利用這一點的示例(使用~
來反轉對正常點的異常值的檢測):
fig, ax = plt.subplots(figsize=(7,5))
non_outliers = dummies1[~((np.abs(dummies1['revenue_zscore']) > THRESHOLD) |
(np.abs(dummies1['profit_zscore']) > THRESHOLD))]
outliers = dummies1[((np.abs(dummies1['revenue_zscore']) > THRESHOLD) |
(np.abs(dummies1['profit_zscore']) > THRESHOLD))]
ax.scatter(non_outliers['revenue'],non_outliers['profit'])
ax.scatter(outliers['revenue'],outliers['profit'], color='red', marker='x')
ax.set_ylabel('Profit')
ax.set_xlabel('Revenue')
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.