使用 Reactor 非阻塞读取文件

Question

Do you know if it's possible to create Flux of file lines really non-blocking way?

The best solution I've found is next:

Flux.using(
           () -> Files.lines(PATH),
           Flux::fromStream,
           Stream::close
 );

Even though it looks non-blocking but it is blocking under the hood.

https://docs.oracle.com/javase/8/docs/api/java/nio/file/Files.html#lines-java.nio.file.Path-java.nio.charset.Charset-

Read all lines from a file as a Stream. Unlike readAllLines, this method does not read all lines into a List, but instead populates lazily as the stream is consumed.

The returned stream encapsulates a Reader.

Is it possible to create Flux from AsynchronousFileChannel?

Thank you in advance

Answer 1

Files.lines(PATH) is completely IO-free. It's part of the NIO (NIO stands for non-blocking I/O or New I/O) Java API. It lazily returns a stream of lines, which is part of .util package and not .io package like BufferedReader .

Answer 2

If you have Spring Framework on classpath, then you can do the following:

import org.springframework.core.codec.StringDecoder;
import org.springframework.core.io.buffer.DataBufferUtils;
import org.springframework.core.io.buffer.DefaultDataBufferFactory;

import java.nio.channels.AsynchronousFileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;

public class AsyncFileRead {
    public static void main(String[] args) {
        StringDecoder stringDecoder = StringDecoder.textPlainOnly();

        DataBufferUtils.readAsynchronousFileChannel(() -> AsynchronousFileChannel.open(Path.of("test/sample.txt"),
                StandardOpenOption.READ), DefaultDataBufferFactory.sharedInstance, 4096)
            .transform(dataBufferFlux -> stringDecoder.decode(dataBufferFlux, null, null, null))
            .blockLast();
    }
}

Alternatively, you can use RxIo library which provides a nice abstraction similar to java.nio.file.Files just with async/reactive support:

import org.javaync.io.AsyncFiles;
import reactor.core.publisher.Flux;

import java.nio.file.Path;

public class AsyncFileRead {
    public static void main(String[] args) {
        Flux.from(AsyncFiles.lines(Path.of("test/sample.txt")))
            .blockLast();
    }
}

Although, it's important to note that even these solutions are not truly non-blocking as depending on the platform (Windows, Linux) AsynchronousFileChannel implementation can be blocking under the hood but at least it delegates that task to a dedicated thread pool.