如何合并两个大的CSV文件？

Question

I have two large .csv files that I would like to join.

file1.csv has the following structure:

productcode; *many useless columns* ; startdate; enddate; *some other useless columns*

file2.csv has the following structure:

productcode; *many useless columns different from file1* ; page; startdate; enddate; *some othe useless columns*

I would like to join the two files into a file (let's say, out.csv ) with the same structure as file1.csv but with the "page" column from file2.csv, ie

productcode; *useless columns* ; page; startdate; enddate; *useless columns*

The join conditions are same productcode and overlapping dates, ie:

file1.productcode == file2.productcode

and

!(file1.endate<file2.startdate or file2.enddate<file1.startdate)

However, I have no idea on how to do that. One possibility could be to export the two CSVs into MySql, process them and then export the result in a final CSV file. However, that takes time (and is resource consuming).

I'm open to any suggestions.

Answer 1

使用pandas加载它们，并使用.join（）函数将两者与所需的列引用结合在一起