Powershell：比较基于2列的2个文件

Question

Good Morning from Germany and sorry for my bad English.

I hope that someone can help me.

We had to compare 2 .xls or .csv documents with over 4000 lines. Both documents have a Column E with the same delivery note number. This delivery note number is not unique, the number can be uses multiple Times in Column E. Each delivery note number also has a number of Pieces in Column D.

If delivery note number and quantity match in both files, we can ignore and delete the line. Comparing two Files with over 4000 Lines is very costly, so i hope that comparing is possible with Powershell an Regular Expression.

My Idea: Convert xls to csv and do the following: Read the Lines and use the Entry of column E and column D. Foreach Entry of Column E, check if this entry exists on the second file. If the Entry exists, check if column D was the same as in file1. If both Entrys match, remove or Copy Both Lines in Both Files.

At least we have two documents with Entrys wich have no assignment.

Is this possible?

With the PowerShell I can handle quite well, but with Regular Expression... :/

Thanks in advance Daniel

Answer 1

If you think of your two values as a composite primary key, it seems to work out. You said first value in column E isn't necessarily unique. Can you tell me if it IS always unique when combined with its quantity?

Regardless, I would recommend, to process this, just to get a unique list of (Col E, col D) combinations, you could even just take a "E,D" formatted string, so long as col E and col D dont contain commas. After getting this unique value, put it in a hashtable with the formatted value as the key, and an array of files for that key as the value.

Now you have a way to efficiently lookup what files exist for what Col E, Col D combination, so you should be able to handle your specific use cases as needed.

Answer 2

4000 lines doesn't sound like a lot. Try this: assuming csv files are called "1.csv" and "2.csv"

add-content 3.csv (get-content 1.csv) 
add-content 3.csv (get-content 2.csv) 

import-csv -header A,B,C,D,E,F 3.csv  | 
    group E,D | 
    where { $_.count -eq 1 } | 
    foreach { $_.group } |
    export-csv 3.diff.csv -noTypeInformation

"3.diff.csv" will contain only unique records.