How to check for duplicates in a list and save each email one by line in a file

Question

I have a list like this:

[
  'test@t-online.de',
  'kontakt@test.de',
  'info@test.de',
  'test@gmx.de',
  'kontakt@test.de',
]

I want to check for duplicates and save the list in a txt file with one email in each line. The final result would be in this case:

      test@t-online.de
      kontakt@test.de
      info@test.de
      test@gmx.de
 

fs.writeFileSync('./results/test.txt', list)

How to do that?

Answer 1

You can perform that by performing a loop through the Array if emails and perform a filter to like bellow

 let emails = [ 'test@t-online.de', 'kontakt@test.de', 'info@test.de', 'test@gmx.de', 'kontakt@test.de', ]; let result = emails.reduce((accumulator, current) => { if(accumulator.indexOf(current) === -1){ accumulator = accumulator.concat(current); } return accumulator; }, []); console.log(result);

Answer 2

Replace list with [...(new Set(list))].join('\n')

• new Set() ensures only duplicates are removed and only unique elements remain.
• join('\n') transforms the Array into a string separated by new-line ( \n ) character.

 const list = [ 'test@t-online.de', 'kontakt@test.de', 'info@test.de', 'test@gmx.de', 'kontakt@test.de', ]; console.log([...(new Set(list))].join('\n'));

Answer 3

for removing duplicated emails:

arr = [...new Set(arr)];

for writing

var writeStream = fs.createWriteStream('./results/test.txt');
writeStream.on('error', function(e) { 
  /* handel error here */ 
});
arr.forEach(email => writeStream.write(email + '\n'));
writeStream.end();

To append data to an existed file Create write stream in append mode

var writeStream = fs.createWriteStream('./results/test.txt', { 'flags': 'a', 'encoding': null, 'mode': 0666});

referhere

Answer 4

Complete code. I have used "a" flag to append the data in same file

let fs = require('fs');

let data = [
    'test@t-online.de',
    'kontakt@test.de',
    'info@test.de',
    'test@gmx.de',
    'kontakt@test.de',
];
let stream = fs.createWriteStream("emails.txt", {'flags': 'a'});
stream.once('open', function(fd) {
    // this will remove duplicates from the array
    const result = data.filter((item, pos) => data.indexOf(item) === pos)
    result.forEach(email => stream.write(email + '\n'));
});