Java：获取输入整数数组的最有效方法

Question

I'm working on a problem that requires me to store a very large amount of integers into an integer array. The input is formatted so that one line displays the amount of integers and the next displays all of the values meant to be stored. Ex:

3
12 45 67

In the problem there is closer to 100,000 integers to be stored. Currently I am using this method of storing the integers:

Scanner scanner = new Scanner(System.in);
int n = scanner.nextInt();

int[] iVau = new int[n];

String[] temp = scanner.nextLine().split(" ");

for(int i = 0; i < n; i++) {
    iVau[i] = Integer.parseInt(temp[i]);
}

This works fine, however the problem I am solving has a strict time limit and my current solution is exceeding it. I know that there is a more efficient way to store this input using buffered readers and input streams, but I don't know how to do it, can someone please show me.

Answer 1

The way you are using Scanner makes your program save a String containing the whole numbers at once, in memory. With 100000 numbers in the 2nd line of your input, it is not so efficient, you could read numbers one after the other without keeping the previous one in memory. So, this way, avoiding using Scanner.readLine() should make your program run faster. You will not have to read the whole line one time, and read a 2nd time this String to parse the integers from it: you will do both of these operations only once.

Here is an example. The method testing() does not use any Scanner. The method testing2() is the one you provided. The file tst.txt contains 100000 numbers. The output from this program, on my Mac Mini (Intel Core i5@2.6GHz) is:

duration without reading one line at a time, without using a Scanner instance: 140 ms
duration when reading one line at a time with a Scanner instance: 198 ms

As you can see, not using Scanner makes your program 41% faster (integer part of (198-140)/140*100 equals 41).

package test1;
import java.io.*;
import java.util.*;

public class Test {
    // Read and parse an Int from the stream: 2 operations at once
    private static int readInt(InputStreamReader ir) throws IOException {
        StringBuffer str = new StringBuffer();
        int c;
        do { c = ir.read(); } while (c < '0' || c > '9');
        do {
            str.append(Character.toString((char) c));
            c = ir.read();
        } while (!(c < '0' || c > '9'));
        return Integer.parseInt(str.toString());
    }

    // Parsing the input step by step
    private static void testing(File f) throws IOException {
        InputStreamReader ir = new InputStreamReader(new BufferedInputStream(new FileInputStream(f)));
        int n = readInt(ir);
        int [] iVau = new int[n];
        for (int i = 0; i < n; i++) iVau[i] = readInt(ir);
        ir.close();
    }

    // Your code
    private static void testing2(File f) throws IOException {
        Scanner scanner = new Scanner(f);
        int n = scanner.nextInt();
        int[] iVau = new int[n];
        scanner.nextLine();     
        String[] temp = scanner.nextLine().split(" ");
        for(int i = 0; i < n; i++)
            iVau[i] = Integer.parseInt(temp[i]);
        scanner.close();
    }

    // Compare durations
    public static void main(String[] args) throws IOException {
        File f = new File("/tmp/tst.txt");          

        // My proposal    
        long t = System.currentTimeMillis();
        testing(f);
        System.out.println("duration without reading one line at a time, without using a Scanner instance: " + (System.currentTimeMillis() - t) + " ms");       

        // Your code    
        t = System.currentTimeMillis();
        testing2(f);
        System.out.println("duration when reading one line at a time with a Scanner instance: " + (System.currentTimeMillis() - t) + " ms");
    }
}

NOTE: creating the input file is done this way, with bash or zsh:

echo 100000 > /tmp/tst.txt
for i in {1..100000}
do
  echo -n $i" " >> /tmp/tst.txt
done

Answer 2

I believe this is what you're looking for. A BufferedReader can only read a line at a time, so it is necessary to split the line and cast String s to int s.

BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

try {
    int n = Integer.parseInt(br.readLine());
    int[] arr = new int[n];

    String[] line = br.readLine().split(" ");
    for (int i = 0; i < n; i++) {
        arr[i] = Integer.parseInt(line[i]);
    }
} catch (IOException e) {
    e.getStackTrace();
}

Answer 3

Just a thought, String.split returns an array of Strings. You say the input can be around 100,000 values. So in order to split the array in this way, String.split must be iterating through each element. Now in parsing the new array of strings to Integers you have iterated through the collection twice. You could do this in one iteration with a few small tweaks.

Scanner scanner = new Scanner(System.in);
String tmp = scanner.nextLine();
scanner = new Scanner(tmp); 

for(int i = 0; scanner.hasNextInt(); i++) {
  arr[i] = scanner.nextInt();
}

The reason for linking the scanner to a String instead of leaving it on System.in is so that it ends properly. It doesn't open System.in for user input on the last token. I believe in big O notation this is the difference between O(n) and O(2n) where the original snippet is O(2n)

Answer 4

I am not quite sure why OP has to use Integer.parseInt(s) here since Scanner can just do the parsing directly by new Scanner(File source) .

Here is a demo/test for this idea:

public class NextInt {
    public static void main(String... args) {
        prepareInputFile(1000, 500); // create 1_000 arrays which each contains 500 numbers;
        Timer.timer(() -> readFromFile(), 20, "NextInt"); // read from the file 20 times using Scanner.nextInt();
        Timer.timer(() -> readTest(), 20, "Split"); // read from the file 20 times using split() and Integer.parseInt();
    }

    private static void readTest() {
        Path inputPath = Paths.get(Paths.get("").toAbsolutePath().toString().concat("/src/main/java/io/input.txt"));
        try (Scanner scanner = new Scanner(new File(inputPath.toString()))) {
            int n = Integer.valueOf(scanner.nextLine());
            int[] iVau = new int[n];
            String[] temp = scanner.nextLine().split(" ");
            for (int i = 0; i < n; i++) {
                iVau[i] = Integer.parseInt(temp[i]);
            }
        } catch (IOException ignored) {
            ignored.printStackTrace();
        }
    }

    private static void readFromFile() {
        Path inputPath = Paths.get(Paths.get("").toAbsolutePath().toString().concat("/src/main/java/io/input.txt"));
        try (Scanner scanner = new Scanner(new File(inputPath.toString()))) {
            while (scanner.hasNextInt()) {
                int arrSize = scanner.nextInt();
                int[] arr = new int[arrSize];
                for (int i = 0; i < arrSize; ++i) {
                    arr[i] = scanner.nextInt();
                }
//                System.out.println(Arrays.toString(arr));
            }
        } catch (IOException ignored) {
            ignored.printStackTrace();
        }
    }

    private static void prepareInputFile(int arrCount, int arrSize) {
        Path outputPath = Paths.get(Paths.get("").toAbsolutePath().toString().concat("/src/main/java/io/input.txt"));
        List<String> lines = new ArrayList<>();
        for (int i = 0; i < arrCount; ++i) {
            int[] arr = new int[arrSize];
            for (int j = 0; j < arrSize; ++j) {
                arr[j] = new Random().nextInt();
            }
            lines.add(String.valueOf(arrSize));
            lines.add(Arrays.stream(arr).mapToObj(String::valueOf).collect(Collectors.joining(" ")));
        }
        try {
            Files.write(outputPath, lines);
        } catch (IOException ignored) {
            ignored.printStackTrace();
        }
    }
}

Locally tested it with 1_000 arrays while each array has 500 numbers, reading all the elements cost about: 340ms using Scanner.nextInt() while OP's method about 1.5ms .

NextInt: LongSummaryStatistics{count=20, sum=6793762162, min=315793916, average=339688108.100000, max=618922475}
Split: LongSummaryStatistics{count=20, sum=26073528, min=740860, average=1303676.400000, max=5724370}

So I really have doubt the issue lies in the input reading.

Answer 5

Since in your case you are aware of the total count of elements all that you have to do is to read X integers from the second line. Here is an example:

public static void main(String[] args) {
        Scanner in = new Scanner(System.in);

        int count = in.nextInt();
        int array[] = new int[count];

        for (int i = 0; i < count; i++) {
            array[i] = in.nextInt();
        }
}

If this is not fast enough, which I doubt, then you could switch to the use of a BufferedReader as follows:

public static void main(String[] args) throws IOException {
        BufferedReader in = new BufferedReader(new InputStreamReader(System.in));

        int count = Integer.parseInt(in.readLine());
        int array[] = new int[count];

        for (int i = 0; i < count; i++) {
            int nextInteger = 0;
            int nextChar = in.read();
            do {
                nextInteger = nextInteger * 10 + (nextChar - '0');
                nextChar = in.read();
            } while (nextChar != -1 && nextChar != (int)' ');
            array[i] = nextInteger;
        }
}

In your case the input will be aways valid so this means that each of the integers will be separated by a single whitespace and the input will end up with EoF character.

If both are still slow enough for you then you could keep looking for more articles about Reading Integers in Java, Competative programming like this one: https://www.geeksforgeeks.org/fast-io-in-java-in-competitive-programming/

Still my favorite language when it comes to competitions will always be C :) Good luck and enjoy!