[英]Python pandas dataframe fill NaN with other Series
I want to fill NaN values in a DataFrame (df) column (var4) based on a control table (fillna_mean) using column mean, and var1 as index.In the dataframe I want them to match on var1. 我想基于控制表(fillna_mean)使用列均值和var1作为索引来填充DataFrame(df)列(var4)中的NaN值。在数据帧中,我希望它们在var1上匹配。
I have tried doing this with fillna but I dont get it to work all the way. 我试过用fillna做这个,但我不能让它一直工作。 How do I do this in a smart way, using df.var1 as index matching fillna_mean.var1? 如何使用df.var1作为索引匹配fillna_mean.var1以智能方式执行此操作?
df: DF:
df = pd.DataFrame({'var1' : list('a' * 3) + list('b' * 2) + list('c' * 4) + list('d' * 3)
,'var2' : [i for i in range(12)]
,'var3' : list(np.random.randint(100, size = 12))
,'var4' : [1, 2, np.nan, 3, 2, np.nan, 1, 34, np.nan, np.nan, 12, 12]
})
fillna_mean: fillna_mean:
fillna = pd.DataFrame({'var1' : ['a', 'b', 'c', 'd'],
'mean' : [1, 3.5, 6.5, 10]})
End result is this: 最终结果如下:
var1 var2 var3 var4 a 0 69 1.0 a 1 17 2.0 a 2 83 1.0 b 3 12 3.0 b 4 36 2.0 c 5 68 6.5 c 6 13 1.0 c 7 30 34.0 c 8 23 6.5 d 9 82 10.0 d 10 32 12.0 d 11 19 12.0
Thanks in advance for input! 提前感谢您的投入!
/swepab / swepab
you can use boolean indexing in conjunction with .map() method: 你可以结合.map()方法使用布尔索引 :
In [178]: fillna.set_index('var1', inplace=True)
In [179]: df.loc[df.var4.isnull(), 'var4'] = df.loc[df.var4.isnull(), 'var1'].map(fillna['mean'])
In [180]: df
Out[180]:
var1 var2 var3 var4
0 a 0 40 1.0
1 a 1 97 2.0
2 a 2 34 1.0
3 b 3 6 3.0
4 b 4 19 2.0
5 c 5 47 6.5
6 c 6 65 1.0
7 c 7 29 34.0
8 c 8 48 6.5
9 d 9 88 10.0
10 d 10 40 12.0
11 d 11 23 12.0
Explanation: 说明:
In [184]: df.loc[df.var4.isnull()]
Out[184]:
var1 var2 var3 var4
2 a 2 75 NaN
5 c 5 75 NaN
8 c 8 44 NaN
9 d 9 34 NaN
In [185]: df.loc[df.var4.isnull(), 'var1']
Out[185]:
2 a
5 c
8 c
9 d
Name: var1, dtype: object
In [186]: df.loc[df.var4.isnull(), 'var1'].map(fillna['mean'])
Out[186]:
2 1.0
5 6.5
8 6.5
9 10.0
Name: var1, dtype: float64
UPDATE: starting from Pandas 0.20.1 the .ix indexer is deprecated, in favor of the more strict .iloc and .loc indexers . 更新:从Pandas 0.20.1开始, .ix索引器已弃用,支持更严格的.iloc和.loc索引器 。
Get faster results with combine_first
, and you don't bother you filter out nonnull data: with combine_first
可以获得更快的结果,并且您不需要过滤掉非空数据:
fillna.set_index('var1', inplace=True)
df.var4 = df.var4.combine_first(df.var1.map(fillna['mean']))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.