Copy columns from Sheet1 to Sheet2 & remote duplicates using Google App Script

Question

How can I Copy/append all unique rows from SOURCE Spreadsheet to DESTINATION Spreadsheet.

• Spreadsheet #1 (SOURCE) - Contains ID's and Names. This sheet has duplicate rows. 500k+ rows (Access: View Only)
• Spreadsheet #2 (DESTINATION) - Contains only unique ID's with Names. (Access: Edit)

---

This script works, but it copies all (including duplicates).

function transferIDs() {
  var sss = SpreadsheetApp.openById('%'); //SOURCE
  var ss = sss.getSheetByName('Sheet1');
  var SRange = ss.getDataRange();
  var A1Range = SRange.getA1Notation();
  var SData = SRange.getValues();

  var dss = SpreadsheetApp.openById('#'); //DESTINATION
  var ds = dss.getSheetByName('Sheet1');
  ds.clear({contentsOnly: true});
  ds.getRange(A1Range).setValues(SData);
}

---

Spreadsheet #1 SOURCE (contains duplicate rows)

A	B
ID	Name
X123456	John
Y112233	Sarah
X998877	Amanda
012344	Bob
X998877	Amanda

Spreadsheet #2 DESTINATION (Populated using GAS, no duplicates, Expected Outcome)

A	B
ID	Name
X123456	John
Y112233	Sarah
X998877	Amanda
012344	Bob

Answer 1

Use filter with set :

Change SData to

const SData = SRange.getValues().filter(
  ((set) => (row) => {
    const jrow = row.join('');
    if (!set.has(jrow)) return set.add(jrow);
  })(new Set())
);

And modify the destination range:

ds.getRange(1,1,SData.length,SData[0].length).setValues(SData);