使用 python/pandas 为 A 列中的每个唯一记录获取 B 列中的唯一值

Question

I'm in search for a quick&productive workaround for the following task.

I need to create a separate column for each DeviceID . The column must contain an array with unique SessionStartDate values for each DeviceID .

For example:

• 8846620190473426378 | [2018-08-01, 2018-08-02]
• 381156181455864495 | [2018-08-01]

Though user 8846620190473426378 may have had 30 sessions on 2018-08-01, and 25 sessions on 2018-08-02, I'm only interested in unique dates when these sessions occurred.

Currently, I'm using this approach:

df_main['active_days'] = [
sorted(
    list(
        set(
            sessions['SessionStartDate'].loc[sessions['DeviceID'] == x['DeviceID']]
            )
        )
    )  
for _, x in df_main.iterrows()
]

df_main here is another DataFrame, containing aggregated data grouped by DeviceID

The approach seems to be very ( Wall time: 1h 45min 58s ) slow, and I believe there's a better solution for the task.

Thanks in advance!

Answer 1

I believe you need sort_values with SeriesGroupBy.unique :

rng = pd.date_range('2017-04-03', periods=4)
sessions = pd.DataFrame({'SessionStartDate': rng, 'DeviceID':[1,2,1,2]})  
print (sessions)
  SessionStartDate  DeviceID
0       2017-04-03         1
1       2017-04-04         2
2       2017-04-05         1
3       2017-04-06         2

#if necessary convert datetimes to dates
sessions['SessionStartDate'] = sessions['SessionStartDate'].dt.date
out = (sessions.sort_values('SessionStartDate')
               .groupby('DeviceID')['SessionStartDate']
               .unique())
print (out)
DeviceID
1    [2017-04-03, 2017-04-05]
2    [2017-04-04, 2017-04-06]
Name: SessionStartDate, dtype: object

Another solution is remove duplicates by drop_duplicates and groupby with converting to list s:

sessions['SessionStartDate'] = sessions['SessionStartDate'].dt.date
out = (sessions.sort_values('SessionStartDate')
               .drop_duplicates(['DeviceID', 'SessionStartDate'])
               .groupby('DeviceID')['SessionStartDate']
               .apply(list))
print (out)
DeviceID
1    [2017-04-03, 2017-04-05]
2    [2017-04-04, 2017-04-06]
Name: SessionStartDate, dtype: object