基于两列值有效地从熊猫数据框中提取信息

Question

I am trying to extract information from a data frame which is indexed by productId and customerId. I have a large number (millions) of (productId, customerId) pairs and am interested in finding the most efficient way possible to do this.

I have two data frames, df1 containing the customerId, productId pairs I'm interested in, and a second frame df2 containing information of interest which is indexed by customerId, productId pairs.

So far I have tried something like:

def f(x, y):
    return(df2.col[(df2.customerId == x) & (df2.productId == y)].sum())

values = df1.apply(lambda x: f(x.customerId, x.productId), axis = 1)

which works fine but is very slow.

Any suggestions on improvements?

Answer 1

您可以尝试列表理解：

values = [df2.loc[df2[['customerId', 'productId']].eq(i).all(), 'col'].sum() for i in df1.values]