使用Java将唯一行写入文件

Question

I have written some code that is almost there in terms of how I want it to function. The logic of this Java code is as follows:

1. Read the source file from the specified location
2. As we're reading each line, apply the regex to get the capture group result (in this instance, the URL)
3. After all these lines are read, put the URL and line number in to the HashMap
4. Copy these values into a list, and order them by line number increasing
5. Read the source file again
6. For each line number matched in the list, write to our new file

And here is the code:

package preproc;

import java.io.*;
import java.util.*;
import java.util.regex.*;

public class Preproc {

    public static void main(String[] args) {

       File file = new File("C:\\Users\\AnthonyH\\Desktop\\file.txt");
       BufferedReader br;

       HashMap<String, Integer> hmap = new HashMap<>();

        try {

            br = new BufferedReader(new InputStreamReader(new FileInputStream(file)));

            int linenumber = 0;
            String event;

            while ((event = br.readLine()) != null) {

        //System.out.println("LINE=" + event);
                Pattern regex = Pattern.compile("^.*url=(.*)");
                Matcher check = regex.matcher(event);
                if (check.find()) {
                    String match = check.group(1);
            //System.out.println("GROUP=" + match + " LINE=" + linenumber);
                    if (!hmap.containsKey(match)) {
            //System.out.println("ADDING TO INDEX");
                        hmap.put(match, linenumber);
                    }
                }

                linenumber++;
            }

            List<Integer> lineNumbers = new ArrayList<>(hmap.values());
        //System.out.println("SIZE=" + lineNumbers.size());
            Collections.sort(lineNumbers);

            File file2 = new File("C:\\Users\\AnthonyH\\Desktop\\file2.txt");
            BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(file)));
            BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file2)));

            int currentLine = 0;

            for (Integer line : lineNumbers) {

        //System.out.println("LINE=" + line + "CURRENT LINE=" + currentLine);
                while (currentLine < line) {
                    reader.readLine();
                    currentLine++;
                }
                writer.write(reader.readLine());
        writer.newLine();
                currentLine++;
            }

        writer.close();

        } catch (IOException e) {

            e.printStackTrace();

        }

    }

}

The issue I'm facing is that it is writing ALL of the unique string matches to the HashMap, when I only want to add those that occur once in the original file. IE five instances of site1.com and one instance of site2.com, the map will have the first instance of site1.com and the unique instance of site2.com. I only would want site2.com.

All help is greatly appreciated.

Answer 1

Create a Map<String, Occurrence> where Occurrence contains the (first) line number, and the number of occurrences of the URL. When writing, ignore the lines for wich the number of occurrences is > 1.

That's one way, there are others.

You could have a Set of urls that are met at least twice. As soon as you find a URL that is already in the map, you add it to the set. When writing, you ignore the URLs that are in the set.

Note that, if the file isn't too large, you could store the lines in memory rather than re-reading the file.

Answer 2

package preproc;

import java.io.*;
import java.util.*;
import java.util.regex.*;

public class Preproc {

public static void main(String[] args) {

   File file = new File("C:\\Users\\AnthonyH\\Desktop\\file.txt");
   BufferedReader br;

   HashMap<String, List<Integer>> hmap = new LinkedHashMap<String, List<Integer>>();

    try {

        br = new BufferedReader(new InputStreamReader(new FileInputStream(file)));

        int linenumber = 0;
        String event;

        while ((event = br.readLine()) != null) {

            Pattern regex = Pattern.compile("^.*url=(.*)");
            Matcher check = regex.matcher(event);
            if (check.find()) {
                String match = check.group(1);

                List<Integer> lineNumbers = new ArrayList<Integer>();
                if (hmap.containsKey(match)) {
                    lineNumbers = hmap.get(match);
                }
                lineNumbers.add(linenumber);

                    hmap.put(match, lineNumbers);
            }

            linenumber++;
        }

        List<List<Integer>> lineNumbers = new ArrayList<List<Integer>>(hmap.values());

        File file2 = new File("C:\\Users\\AnthonyH\\Desktop\\file2.txt");

        BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(file2)));



        for (List<Integer> linesOccurences : lineNumbers) {

            int currentLine = 0;
            if(linesOccurences.size() == 1)
            {
                BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(file)));
                int line = linesOccurences.get(1);
                while (currentLine++ < line) {
                    reader.readLine();
                }
                writer.write(reader.readLine());
                writer.newLine();
                reader.close();
            }

        }

    writer.close();

    } catch (IOException e) {

        e.printStackTrace();

    }

}
}

Try this edited code. In the previous one the BufferedReader object was not at a correct place.