如何使用Java 8流按块读取文本文件

Question

I want to read an ASCII file that is composed of "blocks" that are delimited by start and end tags.

I have never used Java 8 streams and I would like to test them on this file reader, but I don't really know how to do it.

For the sake of simplicity, let's consider the following file format (actual file format can be found here ):

$Node
6
1 1.0 0.0 0.0
2 -1.0 0.0 0.0
3 0.0 1.0 0.0
4 0.0 -1.0 0.0
5 0.0 0.0 1.0
6 0.0 0.0 -1.0
$EndNode
$Elements
3
1 10 1 2 3
2 10 4 5 6
3 10 1 5 3
$EndElements

Where the first line of each block is the number of elements in the block. Then each block is a list of space-separated values. Number of values and types vary depending on the block.

In real life, the file can get pretty big (several hundred Mb, maybe up to a few Gb), so performance is critical.

Using Java NIO 2 (without the Java 8 streams), I would have done something like this:

BufferedReader reader = Files.newBufferedReader(filePath, Charset.defaultCharset());
String line = null;
Parser currentParser = defaultParser;
while ((line = reader.readLine()) != null) {
    if (line.startsWith("$")) {
        currentParser = getParser(line);
        continue;
    }
    currentParser.parseLine(line);
}

With a line parser that would be smart enough to deal with the first line of the block differently that the rest (without having to test a isFirstLineOfBlock boolean for each line)... Don't know yet how to do that either by the way.

Anyway, I would appreciate some help with using Java 8 streams for this file reader.

Final question, what is the advantage of using Java streams for such an application: is it just a question of readability or can I expect improved performances?

Answer 1

You have several ways to do it with Java 8 streams. For example,

try (BufferedReader br = Files.newBufferedReader(Paths.get(filePath, Charset.defaultCharset())) {
    br.lines()
        .filter(line -> !line.startsWith("$"))
        .forEachOrdered(currentParser::parseLine);
} catch (IOException ex) {
    throw new Error(ex);
}

.lines() method description contains

The Stream is lazily populated, i.e., 
read only occurs during the terminal stream operation.

In this example terminal operation is forEachOrdered

---

another one

try (Stream<String> stream = Files.lines(Paths.get(filePath, Charset.defaultCharset())) {
    stream
        .filter(line -> !line.startsWith("$"))
        .forEachOrdered(currentParser::parseLine);
} catch (IOException ex) {
    throw new Error(ex);
}

Answer 2

It's possible to parse such constructs with the help of my free StreamEx library which enhances standard Stream API:

StreamEx.ofLines(filePath, Charset.defaultCharset())
        .groupRuns((a, b) -> !b.startsWith("$"))
        .forEachOrdered(list -> 
            list.subList(1, list.size()).forEach(getParser(list.get(0))::parseLine));

Here we use groupRuns method which combines single file entry into the list. The argument passed to groupRuns is the BiPredicate which applies to the pair of adjacent input elements which should return true if the elements must be grouped. Here we group elements unless the next one starts with "$" . After that we have lazily populated Stream<List<String>> and parse each group creating the parser using the first line and calling parseLine for all subsequent lines.