熊貓基於多個日期加入兩個數據框

Question

我有兩個pandas DataFrames ，我試圖在其中執行高級連接。 在下面的示例中，我想基於my_key加入df和df2 ，其中日期范圍from_dt和to_dt重疊。 如何使用熊貓來做到這一點？

前df：

value, my_key, from_dt, to_dt
1, a, 2007-01-01, 2009-02-01
2, b, 2001-01-01, 2011-01-01
3, c, 2015-01-01, 2020-01-01

df2：

my_key, value2, from_dt, to_dt
a, a1, 2007-01-01, 2008-01-01
a, a2, 2008-01-01, 2010-01-01
b, b1, 2009-01-01, 2015-01-01
c, c1, 2011-01-01, 2011-12-30

預期結果：

value, value2, from_dt, to_dt
1, a1, 2007-01-01, 2008-01-01
1, a2, 2008-01-01, 2009-02-01
2, b1, 2009-01-01, 2011-01-01

Answer 1

@Jianxun的答案很好-還請注意，如果您的數據似乎是CSV格式（問題似乎暗示），則可以使用以下pd.datetime自動獲取pd.datetime

df = pd.read_csv("df.csv", parse_dates=True)

您可能需要查看pandas文檔的這些 部分 。

Answer 2

這可以分兩步完成。 首先進行外部合並，其次保留確實重疊的行。

import pandas as pd

# your data
# ===================================
df

   value my_key     from_dt       to_dt
0      1      a  2007-01-01  2009-02-01
1      2      b  2001-01-01  2011-01-01
2      3      c  2015-01-01  2020-01-01

df2

  my_key value2     from_dt       to_dt
0      a     a1  2007-01-01  2008-01-01
1      a     a2  2008-01-01  2010-01-01
2      b     b1  2009-01-01  2015-01-01
3      c     c1  2011-01-01  2011-12-30

# processing
# ======================================
# outer merge
df_temp = pd.merge(df, df2, on=['my_key'], how='outer')

# just make sure that the columns are in proper datetime type
# you don't have to do this if your data is already in datetime
df_temp.from_dt_x = pd.to_datetime(df_temp.from_dt_x)
df_temp.to_dt_x = pd.to_datetime(df_temp.to_dt_x)
df_temp.from_dt_y = pd.to_datetime(df_temp.from_dt_y)
df_temp.to_dt_y = pd.to_datetime(df_temp.to_dt_y)

   value my_key  from_dt_x    to_dt_x value2  from_dt_y    to_dt_y
0      1      a 2007-01-01 2009-02-01     a1 2007-01-01 2008-01-01
1      1      a 2007-01-01 2009-02-01     a2 2008-01-01 2010-01-01
2      2      b 2001-01-01 2011-01-01     b1 2009-01-01 2015-01-01
3      3      c 2015-01-01 2020-01-01     c1 2011-01-01 2011-12-30

# get rows that do overlap
result = df_temp[(df_temp.to_dt_x >= df_temp.from_dt_y) & (df_temp.from_dt_x <= df_temp.to_dt_y)]

   value my_key  from_dt_x    to_dt_x value2  from_dt_y    to_dt_y
0      1      a 2007-01-01 2009-02-01     a1 2007-01-01 2008-01-01
1      1      a 2007-01-01 2009-02-01     a2 2008-01-01 2010-01-01
2      2      b 2001-01-01 2011-01-01     b1 2009-01-01 2015-01-01

更新：

result['from_dt'] = result[['from_dt_x', 'from_dt_y']].max(axis=1)
result['to_dt'] = result[['to_dt_x', 'to_dt_y']].min(axis=1)
result.drop(['from_dt_x', 'to_dt_x', 'from_dt_y', 'to_dt_y'], axis=1)

   value my_key value2    from_dt      to_dt
0      1      a     a1 2007-01-01 2008-01-01
1      1      a     a2 2008-01-01 2009-02-01
2      2      b     b1 2009-01-01 2011-01-01