[英]Find closest value pairs and calculate mean in Python
我有一个数据框,如下所示:
import pandas as pd
import numpy as np
import random
np.random.seed(5)
df = pd.DataFrame(np.random.randint(100, size=(100, 3)),
columns=list('ABC'),
index=['{}'.format(i) for i in range(100)])
ix = [(row, col) for row in range(df.shape[0]) for col in range(df.shape[1])]
for row, col in random.sample(ix, int(round(.1*len(ix)))):
df.iat[row, col] = np.nan
df = df.mask(np.random.random(df.shape) < .05) #insert 5% of NaNs
df.head()
A B C
0 99 78 61
1 16 73 8
2 62 27 30
3 80 7 76
4 15 53 80
如果我想从columns A, B and C
找到最接近的值对,并计算值对的平均值作为column D
? 我该如何在熊猫中做到这一点? 谢谢。
由于我的真实数据具有某些NaNs
,因此,如果某些行仅具有两个值,则将其均值计算为columns D
;如果某些行仅具有一个值,则将其取入column D
。
我尝试过计算每对的绝对值,从columns diffAB, diffAC and diffBC
找到最小值,然后计算最小对的均值,但我认为这样做更好。
cols = ['A', 'B', 'C']
df[cols]=df[cols].fillna(0)
df['diffAB'] = (df['A'] - df['B']).abs()
df['diffAC'] = (df['A'] - df['C']).abs()
df['diffBC'] = (df['B'] - df['C']).abs()
更新:
df['Count'] = df[['A', 'B', 'C']].apply(lambda x: sum(x.notnull()), axis=1)
if df['Count'] == 3:
def meanFunc(row):
minDiffPairIndex = np.argmin( [abs(row['A']-row['B']), abs(row['B']-row['C']), abs(row['C']-row['A']) ])
meanDict = {0: np.mean([row['A'], row['B']]), 1: np.mean([row['B'], row['C']]), 2: np.mean([row['C'], row['A']])}
return meanDict[minDiffPairIndex]
if df['Count'] == 2:
...
预期结果:
A B C D
0 99 78 61 69.5
1 16 73 8 12
2 62 27 30 28.5
3 80 7 76 78
4 15 53 80 66.5
我在这里使用numpy:
In [11]: x = df.values
In [12]: x.sort()
In [13]: (x[:, 1:] + x[:, :-1])/2
Out[13]:
array([[69.5, 88.5],
[12. , 44.5],
[28.5, 46. ],
[41.5, 78. ],
[34. , 66.5]])
In [14]: np.diff(x)
Out[14]:
array([[17, 21],
[ 8, 57],
[ 3, 32],
[69, 4],
[38, 27]])
In [15]: np.diff(x).argmin(axis=1)
Out[15]: array([0, 0, 0, 1, 1])
In [16]: ((x[:, 1:] + x[:, :-1])/2)[np.arange(len(x)), np.diff(x).argmin(axis=1)]
Out[16]: array([69.5, 12. , 28.5, 78. , 66.5])
In [17]: df["D"] = ((x[:, 1:] + x[:, :-1])/2)[np.arange(len(x)), np.diff(x).argmin(axis=1)]
这可能不是最快的方法,但是非常简单。
def func(x):
a,b,c = x
diffs = np.abs(np.array([a-b,a-c,b-c]))
means = np.array([(a+b)/2,(a+c)/2,(b+c)/2])
return means[diffs.argmin()]
df["D"] = df.apply(func,axis=1)
df.head()
假设您需要具有值对平均值的附加column D
,该值对在(colA, colB), (colB, colC) and (colC, colA)
这三种可能的对中具有最小的差异,以下代码应该可以工作:
更新:
def meanFunc(row):
nonNanValues = [x for x in list(row) if str(x) != 'nan']
numOfNonNaN = len(nonNanValues)
if(numOfNonNaN == 0): return 0
if(numOfNonNaN == 1): return nonNanValues[0]
if(numOfNonNaN == 2): return np.mean(nonNanValues)
if(numOfNonNaN == 3):
minDiffPairIndex = np.argmin( [abs(row['A']-row['B']), abs(row['B']-row['C']), abs(row['C']-row['A']) ])
meanDict = {0: np.mean([row['A'], row['B']]), 1: np.mean([row['B'], row['C']]), 2: np.mean([row['C'], row['A']])}
return meanDict[minDiffPairIndex]
df['D'] = df.apply(meanFunc, axis=1)
上面的代码以以下方式处理行中的NaN
值:如果所有三个值均为NaN
则column D
值为0
;如果两个值为NaN
则将非NaN值分配给column D
;如果正好存在一个NaN
则均值其余两个中的第一个分配给column D
。
以前:
def meanFunc(row):
minDiffPairIndex = np.argmin( [abs(row['A']-row['B']), abs(row['B']-row['C']), abs(row['C']-row['A']) ])
meanDict = {0: np.mean([row['A'], row['B']]), 1: np.mean([row['B'], row['C']]), 2: np.mean([row['C'], row['A']])}
return meanDict[minDiffPairIndex]
df['D'] = df.apply(meanFunc, axis=1)
希望我能正确理解您的问题。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.