Efficient way to find which values in CSV are NOT in DB?

Question

A vendor is feeding us a CSV file of their products. A particular column on the file (eg column 3) is the style number. This file has thousands on entries.

We have a data-base table of products with a column called manufacturer_num which is the vendors style number.

I need to find which of the vendor's products we do not currently have.

I know I can loop throw each line in the CSV file and extract the style_number and check to see if it is in our data-base. But then I am making a call to the data-base for each line. This would be thousands of calls to the data-base. I think this is inefficient.

I could also build a list of the style numbers (either as a string or array) to make one DB call. Something like: WHERE manufactuer_num IN(...) But won't PHP run out of memory if the list is too big? And actually this would give me the ones we do have, not the ones we don't have.

Whats an efficient way to do this?

Answer 1

将CSV批量加载到临时表中，执行LEFT JOIN ，然后获取LEFT JOIN的RHS为NULL 。