逐行将字符串写入文本文件时避免重复

Question

I use the following code to write strings to my simple text file:

EDITED:

private String fileLocation="/mnt/sdcard/out.txt";

public void saveHisToFile()
{
    if (prefs.getBoolean("saveHis", true) && mWordHis != null && mWordHis.size() >= 1)
    {
        StringBuilder sbHis = new StringBuilder();          
        Set<String> wordSet= new HashSet<String>(mWordHis);
        for (String item : wordSet)
        {
            sbHis.append(item);
            sbHis.append("\n");
        }

        String strHis = sbHis.substring(0, sbHis.length()-1);  
        try {
             BufferedWriter bw = new BufferedWriter(new FileWriter(new File(
                     fileLocation), true));
             bw.write(strHis);
             bw.newLine();
             bw.close();
        } catch (IOException e) {

        }
    }
}

The strings are successfully written to the text file, but weirdly, some strings are overwritten, such as:

apple
orange
grapes
grapes
grapes
apple 
kiwi

My question is:

1. how can I stop a string being written more than once?
2. how can I stop writing a string (a line) to the file if it has already existed in the file?

I have consulted this post but failed to apply it to my case. Can you please give a little help? Thanks a lot in advance.

Answer 1

Try this:

public void saveHisToFile(Set<String> existingWords)
{
    if (prefs.getBoolean("saveHis", true) && mWordHis != null && mWordHis.size() >= 1)
    {
        StringBuilder sbHis = new StringBuilder();
        for (String item : mWordHis)
        {
            if (!existingWords.contains(item)) {
                sbHis.append(item);
                sbHis.append("\n");
            }
        }

        String strHis = sbHis.substring(0, sbHis.length()-1);
        try {
            BufferedWriter bw = new BufferedWriter(new FileWriter(new File(
                    fileLocation), true));
            bw.write(strHis);
            bw.newLine();
            bw.close();
        } catch (IOException e) {

        }
    }
}

Answer 2

I guess mWordHis is a List , which can contain duplicate entries. You can first convert it to a Set (which doesn't allow duplicates) and print only the words in the Set .

    Set<String> wordSet= new HashSet<>(mWordHis);

    for (String item : wordSet)
    {
        sbHis.append(item);
        sbHis.append("\n");
    }

As @fge commented, LinkedHashSet may also be used if insertion order matters.

If you need to run the same code several times with the same file, you must either save in memory all the records you've already wrote to the file, or read the file and get all data before writing to it.

Edit:

I can only think about trimming the words as some may contain unneeded spaces:

    Set<String> wordSet= new HashSet<>();
    for (String item : mWordHis){
        wordSet.add(item.trim());
    }

Answer 3

You could add all the strings into a HashMap and check for each new String if it is are already in there.

Example:

    HashMap<String,String> test = new HashMap<String,String>();
    if(!test.containsKey(item)) {
      test.put(item,"");
      // your processing: example
      System.out.println(item);
    } else {
      // Your processing of duplicates, example:
      System.out.println("Found duplicate of: " + item);
    }

Edit: or use a HashSet as shown by the other solutions ...

    HashSet<String> test = new HashSet<String>();
    if(!test.contains(item)) {
      test.add(item);
      // your processing: example
      System.out.println(item);
    } else {
      // Your processing of duplicates, example:
      System.out.println("Found duplicate of: " + item);
    }

Edit2:

    private String fileLocation="/mnt/sdcard/out.txt";

    public void saveHisToFile()
    {
        if (prefs.getBoolean("saveHis", true) && mWordHis != null && mWordHis.size() >= 1)
        {
            StringBuilder sbHis = new StringBuilder();  
            HashSet<String> test = new HashSet<String>();

            Set<String> wordSet= new HashSet<String>(mWordHis);
            for (String item : wordSet)
            {
                if(!test.contains(item)) {
                    test.add(item);
                    // your processing: example
                    sbHis.append(item+System.lineSeparator());
                } else {
                    // Your processing of duplicates, example:
                    //System.out.println("Found duplicate of: " + item);
                }
            }

            String strHis = sbHis.toString();  
            try {
                 BufferedWriter bw = new BufferedWriter(new FileWriter(new File(
                         fileLocation), true));
                 bw.write(strHis);
                 bw.newLine();
                 bw.close();
            } catch (IOException e) {

            }
        }
    }

Answer 4

This is a complete example on how to solve your problem:

      import java.io.BufferedReader;
      import java.io.BufferedWriter;
      import java.io.File;
      import java.io.FileReader;
      import java.io.FileWriter;
      import java.io.IOException;
      import java.util.HashSet;

      public class HisSaver {
        private HashSet<String> uniqueTester = new HashSet<String>();

        private String fileLocation="/mnt/sdcard/out.txt";

        private static HisSaver instance = null;


        private HisSaver() {
            readWordsFromFile();
        }

        public static HisSaver getInstance() {
            if(instance == null)
                instance = new HisSaver();
            return instance;
        }

        public void saveWord(String word) {
            if(!uniqueTester.contains(word)) {
                uniqueTester.add(word);
                writeWordToFile(word);
            }
        }

        private void writeWordToFile(String word) {
          try {
               BufferedWriter bw = new BufferedWriter(new FileWriter(new File(
                       fileLocation), true));
               bw.write(word);
               bw.newLine();
               bw.close();
          } catch (IOException e) {

          }
        }

        private void readWordsFromFile() {
          try {
            BufferedReader br = new BufferedReader(new FileReader(new File(
                fileLocation)));
            String line;
            while((line = br.readLine()) != null) {
                if(!uniqueTester.contains(line)) {
                    uniqueTester.add(line);
                }
            }
          } catch (IOException e) {

          }
        }
      }

Now to use this, you simply do the following in your code:

    HisSaver hs = HisSaver.getInstance();
    hs.saveWord("newWord");

This will insert the "newWord" if and only if it is not already in your file, provided that no other function in your code accesses this file. Please note: this solution is NOT thread safe!!!

Edit: Explanation of what the code does: We create a class HisSaver which is a singleton. This is realized by making it's constructor private and providing a static method getInstance() which returns an initialized HisSaver. This HisSaver will already contain all preexisting words in your file and thus only append new words to it. Calling the getInstance() method from another class will give you a handle for this singleton and allow you to call saveWord without having to worry whether you have the right object in your hands, since only one instance of it can ever be instantiated.