交叉口2熊貓數據幀

Question

在我的問題中我有2個數據幀mydataframe1和mydataframe2如下所示。

mydataframe1
Out[13]:
  Start   End       Remove     
  50      60        1  
  61      105       0  
  106     150       1  
  151     160       0  
  161     180       1  
  181     200       0  
  201     400       1  


mydataframe2
Out[14]: 
    Start   End  
    55      100
    105     140
    151     154
    155     185
    220     240    

從mydataframe2我想在mydataframe2中的任何"Remove" = 1間隔中"Remove"包含間隔Start-End（也部分）的mydataframe1 。 換句話說，不應該有的間隔之間的任何itnersection mydataframe2和每個間隔的mydataframe1

在這種情況下，mydataframe2成為

mydataframe2
Out[15]: 
    Start   End  
    151     154

Answer 1

您可以使用pd.IntervalIndex進行交叉

獲取要刪除的行

In [313]: dfr = df1.query('Remove == 1')

構造IntervalIndex以刪除范圍

In [314]: s1 = pd.IntervalIndex.from_arrays(dfr.Start, dfr.End, 'both')

構造要測試的IntervalIndex

In [315]: s2 = pd.IntervalIndex.from_arrays(df2.Start, df2.End, 'both')

選擇不在s1范圍內的s2行

In [316]: df2.loc[[x not in s1 for x in s2]]
Out[316]:
   Start  End
2    151  154

---

細節

In [320]: df1
Out[320]:
   Start  End  Remove
0     50   60       1
1     61  105       0
2    106  150       1
3    151  160       0
4    161  180       1
5    181  200       0
6    201  400       1

In [321]: df2
Out[321]:
   Start  End
0     55  100
1    105  140
2    151  154
3    155  185
4    220  240

In [322]: dfr
Out[322]:
   Start  End  Remove
0     50   60       1
2    106  150       1
4    161  180       1
6    201  400       1

IntervalIndex詳細信息

In [323]: s1
Out[323]:
IntervalIndex([[50, 60], [106, 150], [161, 180], [201, 400]]
              closed='both',
              dtype='interval[int64]')

In [324]: s2
Out[324]:
IntervalIndex([[55, 100], [105, 140], [151, 154], [155, 185], [220, 240]]
              closed='both',
              dtype='interval[int64]')

In [326]: [x not in s1 for x in s2]
Out[326]: [False, False, True, False, False]

Answer 2

我們可以使用面向中間或長度的樹：重疊測試 ：

In [143]: d1 = d1.assign(s=d1.Start+d1.End, d=d1.End-d1.Start)

In [144]: d2 = d2.assign(s=d2.Start+d2.End, d=d2.End-d2.Start)

In [145]: d1
Out[145]:
   Start  End  Remove    d    s
0     50   60       1   10  110
1     61  105       0   44  166
2    106  150       1   44  256
3    151  160       0    9  311
4    161  180       1   19  341
5    181  200       0   19  381
6    201  400       1  199  601

In [146]: d2
Out[146]:
   Start  End   d    s
0     55  100  45  155
1    105  140  35  245
2    151  154   3  305
3    155  185  30  340
4    220  240  20  460

現在我們可以檢查重疊間隔和過濾器：

In [148]: d2[~d2[['s','d']]\
     ...:       .apply(lambda x: ((d1.loc[d1.Remove==1, 's'] - x.s).abs() <
     ...:                         d1.loc[d1.Remove==1, 'd'] +x.d).any(),
     ...:              axis=1)]\
     ...:   .drop(['s','d'], 1)
     ...:
Out[148]:
   Start  End
2    151  154

Answer 3

我認為這應該有效：

mydataframe2[mydataframe2.Start.isin(mydataframe1[mydataframe1.Remove != 0].Start)]

打破它：

# This filter will remove anything which has Remove not 0
filter_non_remove = mydataframe1.Remove != 0

# This provides a valid Sequence of Start values
valid_starts = mydataframe1[mydataframe1.Remove != 0].Start

# Another filter, that checks whether the Start 
# value is in the valid_starts Sequence
is_df2_valid = mydataframe2.Start.isin(valid_starts)

# Final applied filter
output = mydataframe2[is_df2_valid]

Answer 4

您可以從標記為Remove的列中獲取所有唯一范圍值，然后評估mydataframe2中包含的Start日期和End日期不在任何范圍值中。 第一部分將定義所有唯一值，其中Start / End值為Remove = 1。

start_end_remove = mydataframe1[mydataframe1['Remove'] == 1][['Start', 'End']].as_matrix()
remove_ranges = set([])
for x in start_end_remove:
    remove_ranges.update(np.arange(x[0], x[1] + 1))

接下來，您可以針對唯一的范圍值集評估mydataframe2 。 如果mydataframe2的開始/結束值在值范圍內，則通過標記是否應在新列中刪除它們來從數據框中刪除它們。 定義函數以查看任何范圍之間是否存在重疊，然后將該函數應用於mydataframe2每一行並刪除范圍重疊的行。

def evaluate_in_range(x, remove_ranges):
    s = x[0]
    e = x[1]
    eval_range = set(np.arange(s, e + 1))
    if len(eval_range.intersection(remove_ranges)) > 0:
        return 1
    else:
        return 0

mydataframe2['Remove'] = mydataframe2[['Start', 'End']].apply(lambda x: evaluate_in_range(x, remove_ranges), axis=1)
mydataframe2.drop(mydataframe2[mydataframe2['Remove'] == 1].index, inplace=True)

Answer 5

這個怎么樣：

mydataframe1['key']=1
mydataframe2['key']=1

df3 = mydataframe2.merge(mydataframe1, on="key")

df3['s_gt_s'] = df3.Start_y > df3.Start_x
df3['s_lt_e'] = df3.Start_y < df3.End_x
df3['e_gt_s'] = df3.End_y > df3.Start_x
df3['e_lt_e'] = df3.End_y < df3.End_x

df3['s_in'] = df3.s_gt_s & df3.s_lt_e
df3['e_in'] = df3.e_gt_s & df3.e_lt_e

df3['overlaps'] = df3.s_in | df3.e_in

my_new_dataframe = df3[df3.overlaps & df3.Remove==1][['End_x','Start_x']].drop_duplicates()