如何讓pandas merge_asof 包含所有事件而不僅僅是一個

Question

我有一個包含班次開始和結束的數據集，如下所示：

schedule = pd.DataFrame({
    "start": pd.to_datetime(['2017-01-01 00:59:00', '2017-01-01 04:59:00', '2017-01-02 00:59:00', '2017-01-02 08:00:00', '2017-01-02 09:59:00']),
    "end": pd.to_datetime(['2017-01-01 09:59:00', '2017-01-01 18:00:00', '2017-01-02 09:59:00', '2017-01-02 15:59:00', '2017-01-02 18:00:00']),
    "employee": ['KC', 'IT', 'ED', 'NK', 'IT']
})

我希望最終能夠知道有多少人（和誰）在一天中的特定時間工作。 所以我嘗試使用我想要的頻率的時間戳制作一個新的 DataFrame：

shifts = pd.DataFrame()
shifts['timestamp'] = pd.date_range(start=schedule.start.min(), end=schedule.end.max(), freq='2H')

並[有條件地]將其與我的原始時間表合並，如下所示：

mrg = pd.merge_asof(shifts, schedule, left_on='timestamp', right_on='start').query('timestamp <= end')

結果如下所示：

timestamp                   employee   end         start
0 2017-01-01 00:59:00       KC 2017-01-01 09:59:00 2017-01-01 00:59:00   
1 2017-01-01 02:59:00       KC 2017-01-01 09:59:00 2017-01-01 00:59:00   
2 2017-01-01 04:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00   
3 2017-01-01 06:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00   
4 2017-01-01 08:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00   
5 2017-01-01 10:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00

現在我的問題是，在 2017-01-01 04:59:00 和 2017-01-01 09:59:00 之間的時間戳，當 KC 和 IT 都在工作時，但 mrg 數據框只保留與 IT 相對應的行。 為什么會這樣，我發送給我的 merge_asof 的參數中缺少什么？

Answer 1

似乎您需要將所有employees與timestamps結合起來，然后by以下方式添加參數：

from  itertools import product

t = pd.date_range(start=schedule.start.min(), end=schedule.end.max(), freq='2H')
e = schedule['employee'].unique().tolist()
shifts = pd.DataFrame(list(product(t,e)), columns=['timestamp','employee'])
print (shifts.head(10))
            timestamp employee
0 2017-01-01 00:59:00       KC
1 2017-01-01 00:59:00       IT
2 2017-01-01 00:59:00       ED
3 2017-01-01 00:59:00       NK
4 2017-01-01 02:59:00       KC
5 2017-01-01 02:59:00       IT
6 2017-01-01 02:59:00       ED
7 2017-01-01 02:59:00       NK
8 2017-01-01 04:59:00       KC
9 2017-01-01 04:59:00       IT

---

mrg = pd.merge_asof(shifts, 
                    schedule, 
                    left_on='timestamp', 
                    right_on='start', 
                    by='employee').query('timestamp <= end')
print (mrg)
             timestamp employee                 end               start
0  2017-01-01 00:59:00       KC 2017-01-01 09:59:00 2017-01-01 00:59:00
4  2017-01-01 02:59:00       KC 2017-01-01 09:59:00 2017-01-01 00:59:00
8  2017-01-01 04:59:00       KC 2017-01-01 09:59:00 2017-01-01 00:59:00
9  2017-01-01 04:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00
12 2017-01-01 06:59:00       KC 2017-01-01 09:59:00 2017-01-01 00:59:00
13 2017-01-01 06:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00
16 2017-01-01 08:59:00       KC 2017-01-01 09:59:00 2017-01-01 00:59:00
17 2017-01-01 08:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00
21 2017-01-01 10:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00
25 2017-01-01 12:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00
29 2017-01-01 14:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00
33 2017-01-01 16:59:00       IT 2017-01-01 18:00:00 2017-01-01 04:59:00
50 2017-01-02 00:59:00       ED 2017-01-02 09:59:00 2017-01-02 00:59:00
54 2017-01-02 02:59:00       ED 2017-01-02 09:59:00 2017-01-02 00:59:00
58 2017-01-02 04:59:00       ED 2017-01-02 09:59:00 2017-01-02 00:59:00
62 2017-01-02 06:59:00       ED 2017-01-02 09:59:00 2017-01-02 00:59:00
66 2017-01-02 08:59:00       ED 2017-01-02 09:59:00 2017-01-02 00:59:00
67 2017-01-02 08:59:00       NK 2017-01-02 15:59:00 2017-01-02 08:00:00
69 2017-01-02 10:59:00       IT 2017-01-02 18:00:00 2017-01-02 09:59:00
71 2017-01-02 10:59:00       NK 2017-01-02 15:59:00 2017-01-02 08:00:00
73 2017-01-02 12:59:00       IT 2017-01-02 18:00:00 2017-01-02 09:59:00
75 2017-01-02 12:59:00       NK 2017-01-02 15:59:00 2017-01-02 08:00:00
77 2017-01-02 14:59:00       IT 2017-01-02 18:00:00 2017-01-02 09:59:00
79 2017-01-02 14:59:00       NK 2017-01-02 15:59:00 2017-01-02 08:00:00
81 2017-01-02 16:59:00       IT 2017-01-02 18:00:00 2017-01-02 09:59:00

Answer 2

conditional_join從pyjanitor可以以抽象/便利是有用的：

# pip install pyjanitor
import pandas as pd
import janitor

shifts.conditional_join(
        schedule, 
        ('timestamp', 'start', '>='), 
        ('timestamp', 'end', '<=')
    )

    timestamp               start                 end           employee
0  2017-01-01 00:59:00 2017-01-01 00:59:00 2017-01-01 09:59:00       KC
1  2017-01-01 02:59:00 2017-01-01 00:59:00 2017-01-01 09:59:00       KC
2  2017-01-01 04:59:00 2017-01-01 00:59:00 2017-01-01 09:59:00       KC
3  2017-01-01 04:59:00 2017-01-01 04:59:00 2017-01-01 18:00:00       IT
4  2017-01-01 06:59:00 2017-01-01 00:59:00 2017-01-01 09:59:00       KC
5  2017-01-01 06:59:00 2017-01-01 04:59:00 2017-01-01 18:00:00       IT
6  2017-01-01 08:59:00 2017-01-01 00:59:00 2017-01-01 09:59:00       KC
7  2017-01-01 08:59:00 2017-01-01 04:59:00 2017-01-01 18:00:00       IT
8  2017-01-01 10:59:00 2017-01-01 04:59:00 2017-01-01 18:00:00       IT
9  2017-01-01 12:59:00 2017-01-01 04:59:00 2017-01-01 18:00:00       IT
10 2017-01-01 14:59:00 2017-01-01 04:59:00 2017-01-01 18:00:00       IT
11 2017-01-01 16:59:00 2017-01-01 04:59:00 2017-01-01 18:00:00       IT
12 2017-01-02 00:59:00 2017-01-02 00:59:00 2017-01-02 09:59:00       ED
13 2017-01-02 02:59:00 2017-01-02 00:59:00 2017-01-02 09:59:00       ED
14 2017-01-02 04:59:00 2017-01-02 00:59:00 2017-01-02 09:59:00       ED
15 2017-01-02 06:59:00 2017-01-02 00:59:00 2017-01-02 09:59:00       ED
16 2017-01-02 08:59:00 2017-01-02 00:59:00 2017-01-02 09:59:00       ED
17 2017-01-02 08:59:00 2017-01-02 08:00:00 2017-01-02 15:59:00       NK
18 2017-01-02 10:59:00 2017-01-02 08:00:00 2017-01-02 15:59:00       NK
19 2017-01-02 10:59:00 2017-01-02 09:59:00 2017-01-02 18:00:00       IT
20 2017-01-02 12:59:00 2017-01-02 08:00:00 2017-01-02 15:59:00       NK
21 2017-01-02 12:59:00 2017-01-02 09:59:00 2017-01-02 18:00:00       IT
22 2017-01-02 14:59:00 2017-01-02 08:00:00 2017-01-02 15:59:00       NK
23 2017-01-02 14:59:00 2017-01-02 09:59:00 2017-01-02 18:00:00       IT
24 2017-01-02 16:59:00 2017-01-02 09:59:00 2017-01-02 18:00:00       IT

這將返回時間戳在開始和結束之間的行。 如果間隔不重疊，更有效的解決方案是使用pd.IntervalIndex 。