Pandas dataframe 列值取决于动态行数

Question

I've got a dataframe that looks something like this:

user	current_date	prior_date	points_scored
1	2021-01-01	2020-10-01	5
2	2021-01-01	2020-10-01	4
2	2021-01-21	2020-10-21	4
2	2021-05-01	2021-02-01	4

The prior_date column is simply current_date - 3 months and points_scored is the number of points scored on current_date . I'd like to be able to identify which rows had sum(points_scored) >= 8 where for a given user , the rows considered would be where current_date is between current_date and prior_date . It is guaranteed that no single row will have a value of points_scored >= 8 .

For example, in the example above, I'd like something like this returned:

user	current_date	prior_date	points_scored	flag
1	2021-01-01	2021-04-01	5	0
2	2021-01-01	2020-10-01	4	0
2	2021-01-21	2020-10-21	4	1
2	2021-05-01	2021-02-01	4	0

The third row shows flag=1 because for row 3's values of current_date=2021-01-21 and prior_date=2020-10-21 , the rows to consider would be rows 2 and 3. We consider row 2 because row 2's current_date=2021-01-01 which is between row 3's current_date and prior_date .

Ultimately, I'd like to end up with a data structure where it shows distinct user and flag. It could be a dataframe or a dictionary-- anything easily referencable.

user	flag
1	0
2	1

To do this, I'm doing something like this:

flags = {}
ids = list(df['user'].value_counts()[df['user'].value_counts() > 2].index)
for id in ids:
    temp_df = df[df['user'] == id]
    for idx, row in temp_df.iterrows():
        cur_date = row['current_date']
        prior_date = row['prior_date']
        temp_total = temp_df[(temp_df['current_date'] <= cur_date) & (cand_df['current_date'] >= prior_date)]['points_scored'].sum()
        if temp_total >= 8:
            flags[id] = 1
            break

The code above works, but just takes way too long to actually execute.

Answer 1

You are right, performing loops on large data can be quite time consuming. This is where the power of numpy comes into full play. I am still not sure of what you want but i can help address the speed Numpy.select can perform your if else statement efficiently.

import pandas as pd 
import numpy as np

condition = [df['points_scored']==5, df['points_scored']==4, df['points_scored'] ==3] # <-- put your condition here
choices = ['okay', 'hmmm!', 'yes'] #<--what you want returned (the order is important)
np.select(condition,choices,default= 'default value')

Also, you might want to more succint what you want. meanwhile you can refactor your loops with np.select()