[英]For loop equivalent to reduce execution time in pandas data frame operation
I have written a for loop like this: 我写了这样的for循环:
for i in newc2sdf.Source.unique():
ydf=newc2sdf[newc2sdf.Source==i]
for j in newc2sdf.Destination.unique():
ydf1=ydf[ydf.Destination==j]
As I have so many unique records, It takes huge time to execute. 由于我有很多独特的记录,因此执行需要花费大量时间。
I will do some basic operations from the ydf1 and it will return a single value and I will append the value in a list. 我将从ydf1做一些基本操作,它将返回一个值,并将该值附加在列表中。
And I want to calculate the sum of values from another column where the source and destinations will be unique. 我想从源和目标将是唯一的另一列计算值的总和。
I have another column called timestamp (ex: 2016-08-01 00:10:01) and it's in numpy.datetime64 format, I want the sum of those where the timestamp will be 5 minutes more than the minimum timestamp for a particular source to destination. 我还有另一列称为timestamp的列(例如:2016-08-01 00:10:01),它采用numpy.datetime64格式,我希望这些值的总和比特定来源的最小时间戳长5分钟到目的地。
Is there any alternatives to reduce the execution time. 是否有其他选择可以减少执行时间。
Given the following sample dataframe: 给定以下示例数据框:
newc2sdf = pd.DataFrame([['Home','Seattle',3],['Vacation','San Francisco',74],['Work','Portland',9],
['Vacation','Seattle',24],['Work','Portland',4],['Home','Seattle',5],
['Work','Portland',31],['Vacation','San Francisco',19],['Work','San Francisco',38],
['Home','Seattle',85],['Work','San Francisco',32],['Vacation','Seattle',73]],
columns=['Source','Destination','Value'])
Which gives: 这使:
Source Destination Value
0 Home Seattle 3
1 Vacation San Francisco 74
2 Work Portland 9
3 Vacation Seattle 24
4 Work Portland 4
5 Home Seattle 5
6 Work Portland 31
7 Vacation San Francisco 19
8 Work San Francisco 38
9 Home Seattle 85
10 Work San Francisco 32
11 Vacation Seattle 73
To calculate "the sum of values from another column where the source and destinations will be unique", I would imagine you are looking for groupby()
and agg()
: 要计算“源和目标将是唯一的另一列中的值之和”,我可以想象您正在寻找
groupby()
和agg()
:
newc2sdf.groupby(['Source','Destination']).agg({'Value': 'sum'}))
Yields: 产量:
Value
Source Destination
Home Seattle 93
Vacation San Francisco 93
Seattle 97
Work Portland 44
San Francisco 70
And finally, if you wanted to store this column of values to a list: 最后,如果您要将值的此列存储到列表中:
newc2sdf.groupby(['Source','Destination']).agg({'Value': 'sum'})['Value'].tolist()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.