使用Java 7 nio读写文件

Question

I have files which consist of json elements in an array. (several file. each file has json array of elements)

I have a process that knows to take each json element as a line from file and process it.

So I created a small program that reads the JSON array, and then writes the elements to another file. The output of this utility will be the input of the other process.

I used Java 7 NIO (and gson).

I tried to use as much Java 7 NIO as possible. Is there any improvement I can do? What about the filter? Which approach is better?

Thanks,

public class TransformJsonsUsers {

public TransformJsonsUsers() {
}

public static void main(String[] args) throws IOException {
    final Gson gson = new Gson();

    Path path = Paths.get("C:\\work\\data\\resources\\files");
    final Path outputDirectory = Paths
            .get("C:\\work\\data\\resources\\files\\output");

    DirectoryStream.Filter<Path> filter = new DirectoryStream.Filter<Path>() {

        @Override
        public boolean accept(Path entry) throws IOException {
            // which is better?
            // BasicFileAttributeView attView = Files.getFileAttributeView(entry, BasicFileAttributeView.class);
           // return attView.readAttributes().isRegularFile();
            return !Files.isDirectory(entry);
        }

    };

    DirectoryStream<Path> directoryStream = Files.newDirectoryStream(path, filter);

    directoryStream.forEach(new Consumer<Path>() {
        @Override
        public void accept(Path filePath) {
            String fileOutput = outputDirectory.toString() + File.separator + filePath.getFileName();
            Path fileOutputPath = Paths.get(fileOutput);
            try {
                BufferedReader br = Files.newBufferedReader(filePath);
                User[] users = gson.fromJson(br, User[].class);
                BufferedWriter writer = Files.newBufferedWriter(fileOutputPath, Charset.defaultCharset());
                for (User user : users) {
                    writer.append(gson.toJson(user));
                    writer.newLine();
                }
                writer.flush();
            } catch (IOException e) {
                throw new RuntimeException(filePath.toString(), e);
            }

        }
    });
}

}

Answer 1

There is no point of using Filter if you want to read all the files from the directory. Filter is primarily designed to apply some filter criteria and read a subset of files. Both of them may not have any real difference in over all performance.

If you looking to improve performance, you can try couple different approaches.

Multi-threading

Depending on how many files exists in the directory and how powerful your CPU is, you can apply multi threading to process more than one file at a time

Queuing

Right now you are reading and writing to another file synchronously. You can queue content of the file using Queue and create asynchronous writer.

You can combine both of these approaches as well to improve performance further.

Answer 2

Don't put the I/O into the filter. That's not what it's for. You should get the complete list of files and then process it. For example if the I/O creates another file in the directory, the behaviour is undefined. You might miss a file, or see the new file in the accept() method.