LiNQ Compare two datatable

Question

This question has been asked before, BUT this specific part has not been. I have two csv files, 1.csv and 2.csv.

I first covert them both into DataTable They both have the same schema:

1.csv
ID SUBID QTY Value
a  1     55  TEN
a  2     5   FIFTH
b  1     2   TE

I need to compare the table above with primary keys {ID, SUBID} mapped with table below with QTY having tolerance of 1 digit off:

2.csv
ID SUBID QTY Value
a  1     5  TEN
a  2     6  FIFTH
b  1     2  TEN

The output should only be the difference datatable and output of the 2.csv values

output.csv
ID SUBID QTY Value
a  1     5  
b  1        TEN

Of course this could be done by reading all the values in 1.csv and having a dictionary of {ID, SUBID} -> {QTY, Value} and match that way, but this is a huge list and the time complexity is going to be huge. I was wondering if there is a way to do this through LiNQ ie get 1.csv {ID,SUBID} match with 2.csv {ID,SUBID} and traverse through the data table if any mismatch of {QTY - tolerance of over 1 # OR str(Value) difference} put it in a new datatable.

Answer 1

I would use FileHelpers to load your csv's into a custom class, then you can do this:

var inner=csv1.Join(csv2,
    c1=>new {c1.ID,c1.SUBID},
    c2=>new {c2.ID,c2.SUBID},
    (c1,c2)=>new {c1,c2}).Where(c=>c.c1.Value!=c.c2.Value || Math.Abs(c1.QTY-c2.QTY)>1)
  .Select(c=>new {
    c2.ID,
    c2.SUBID,
    QTY=(c.c1.QTY==c.c2.Value)?null,c.c2.Value,
    Value=c.c1.Value==c.c2.Value?null,c.c2.Value);

using MoreLinq , you can do this:

var left=csv1.ExceptBy(csv2,c=>new {c.ID,c.SUBID})
  .Select(c=>new {c.ID,c.SUBID,QTY=null,Value=null});
var right=csv2.ExceptBy(csv1,c=>new {c.ID,c.SUBID});

var combined=inner.Concat(left).Concat(right);

Then use FileHelper to write out your combined result back to a CSV.

If you know that all combinations of ID,SUBID that are in csv1 are in csv2, and all the combinations that are in csv2 are also in csv1, then you don't need the left/right and can just output what is in inner.