How to get (some) filenames within a directory efficiently

Question

I have a directory with 20M+ files in it. If I try

folder.list();

It'll take me about 5~10min to get the file (sometimes even more). I don't need all file names, just a handful everytime.

In Linux if I try:

• ls -l | head -n 100 ls -l | head -n 100 : it'll take forever
• ls -f | head -n 100 ls -f | head -n 100 : response is obtained in a few seconds.

So I can list files quickly if I use ProcessBuilder and run something like ls -f | head -n 100 ls -f | head -n 100

Is there a native way for listing a fixed number of files within a directory without requiring to using ProcessBuilder ?

Answer 1

Yes, there is way using Java NIO.2 and the class DirectoryStream . This class implements a lazy iterable over the entries of a directory. You can obtain an instance of DirectoryStream<Path> using Files.newDirectoryStream(path) (and you can obtain an instance of Path with the static factories Paths.get ).

If you are using Java 8, an even simpler solution would be to use Files.list() :

Return a lazily populated Stream , the elements of which are the entries in the directory. The listing is not recursive.

You could then have

List<String> fileNames = Files.list(path)
                              .map(Path::getFileName)
                              .map(Path::toString)
                              .limit(100)
                              .collect(Collectors.toList());

to retrieve the 100 file names of the given path .

Answer 2

you could try if the java7 nio file operations are faster for you. they were a big improvement in one case where i used them: https://docs.oracle.com/javase/tutorial/essential/io/walk.html