将两个熊猫数据框与条件合并

Question

I have two dataframes that I need to merge on complex conditions. Here the two dataframe:

     dock_id     dock_name               avail_bikes    avail_docks  \
0    3082        Hope St & Union Ave     8              16   
1    468         Broadway & W 55 St      0              59   
2    407         Henry St & Poplar St    22             15   
3    3016        Kent Ave & N 7 St       29             16   

    status_key   datehour             ...    visi  vism   wdird     wdire  \
0   1            2016-06-01 19:25:00  ...    NaN   NaN    NaN       NaN   
1   1            2016-06-01 19:25:00  ...    NaN   NaN    NaN       NaN   
2   1            2016-06-01 19:25:00  ...    NaN   NaN    NaN       NaN   
3   1            2016-06-01 19:25:00  ...    NaN   NaN    NaN       NaN    

     tot_docks    _lat               _long         in_service  
0    25           40.711674          -73.951413    1   
1    59           40.765265          -73.981923    1   
2    37           40.700469          -73.991454    1   
3    47           40.720368          -73.961651    1   

....

        Start Date/Time         End Date/Time            Event Agency  \
0       01/01/2016 12:00:00 AM  01/01/2016 02:00:00 AM   Parks Department   
1       01/02/2016 12:00:00 AM  01/02/2016 02:00:00 AM   Parks Department   
2       01/03/2016 12:00:00 AM  01/03/2016 02:00:00 AM   Parks Department   
3       01/04/2016 12:00:00 AM  01/04/2016 02:00:00 AM   Parks Department   

        latitude   longitude  
0       40.782865  -73.965355  
1       40.782865  -73.965355  
2       40.782865  -73.965355  
3       40.782865  -73.965355  
4       40.782865  -73.965355 

I would like to join them with the conditions:

Start Date/Time <= datehour <= End Date/Time and distance(_lat,_lon,latitude,longitude) < d

I know it is possible to merge data and then apply a filter on it to do it but the datasets are too big (10263241 rows and 401080 rows). So I don't think this method would work in a reasonable time.

Have you any idea how could we solve this problem?

Thanks for your answers!

Answer 1

import pandas as pd ... new_frame=pd.merge(dataframe1,dataframe2,on condition)

in case of more advanced merge we can specify the column names as well dataframe[['column1','column2',...]]