Recursively process same files inside many sub directories in Java

Question

Hi I would like to process files inside many sub directories using Java. Psuedo code would be

while(mainDir.hasMoreDirectory())
{
   getFilesFromCurrentDirectory()
   passThoseFilesAsArgumentToProcess()
}

I am currently using the following code

public void list(File file) {
    System.out.println(file.getName());
    File[] children = file.listFiles();
    for (File child : children) {
        list(child);
    }
}

Above code just lists files. Other thing I can do is I have to store list of files and directories in a list and then process in another loop. But I am not able to come up with what I want as show in pseudo code. I am new to Files Directories please help. Thanks in advance.

Answer 1

If you are using Java 7, you can harness the enhanced functionality of NIO in the form of the Files.walkFileTree method. Traversing the file system has never been easier in Java.

There is a short tutorial on it's usage here .

It implements the visitor pattern so you don't need to worry about the traversal algorithm itself, only specify what you want to do with each entry.

Answer 2

When traveling a directory tree in Java 7 use the Paths and Files functionality. They not only ease reading of directories and files, they're way faster then the "old" File way.

Assume you have two directories: mainDir and otherDir and you want to walk thru all directories of mainDir down to its leaves. With each entry in maiondir (file, sub-directory, symbolic link, ...) you want to compare this entry and its attributes (size, modification time, ...) against the entry at the same position in the otherDir . Then this would be your code:

public final void test() throws IOException, InterruptedException {
    final Path mainDir = Paths.get("absolute path to your main directory to read from");
    final Path otherDir = Paths.get("absolute path to your other directory to compare");

    // Walk thru mainDir directory
    Files.walkFileTree(mainDir, new FileVisitor<Path>() {
        @Override
        public FileVisitResult preVisitDirectory(Path path,
                BasicFileAttributes atts) throws IOException {
            return visitFile(path, atts);
        }

        @Override
        public FileVisitResult visitFile(Path path, BasicFileAttributes mainAtts)
                throws IOException {
            // I've seen two implementations on windows and MacOSX. One has passed the relative path, one the absolute path.
            // This works in both cases
            Path relativePath = mainDir.relativize(mainDir.resolve(path));

            BasicFileAttributes otherAtts = Files.readAttributes(otherDir.resolve(relativePath), BasicFileAttributes.class);

            // Do your comparison logic here:
            compareEntries(mainDir, otherDir, relativePath, mainAtts, otherAtts);

            return FileVisitResult.CONTINUE;
        }

        @Override
        public FileVisitResult postVisitDirectory(Path path,
                IOException exc) throws IOException {
            // TODO Auto-generated method stub
            return FileVisitResult.CONTINUE;
        }

        @Override
        public FileVisitResult visitFileFailed(Path path, IOException exc)
                throws IOException {
            exc.printStackTrace();

            // If the root directory has failed it makes no sense to continue
            return (path.equals(mainDir))? FileVisitResult.TERMINATE:FileVisitResult.CONTINUE;
        }
    });
}

What it not does:

• Find entries that do exist in otherDir but not in maindir
• Path and BasicFileAttributes are not Serializable , so there's no easy way to do this walk on two different machines.

Answer 3

Will following do?

public void list(File file) {
    File[] children = file.listFiles();

    if (children != null) {
        process(children);

        for (File child : children) {
            if (child.isDirectory()) {
                list(child);
            }
        }
    } else {
        process(new File[]{file});
    }
}

private void process(File[] children) {
    for (File child : children) {
        if (child.isFile()) {
            // process normal file
        }
    }
}

Answer 4

A method like this would return you a List of all the files recursively within a directory. You can either operate on the returned List or replace the rtn.add calls with your processing.

Beware that this method doesn't have anything to stop it getting stuck in circular symlinks.

public static List<File> getFilesRecursive(File s)
{
    ArrayList<File> rtn = new ArrayList<File>();
    File[] contents = s.listFiles();
    for(int i = 0; i<contents.length; i++)
    {
        if(contents[i].isDirectory()){
            rtn.addAll(getFilesRecursive(contents[i]));
        }else{
            rtn.add(contents[i]);
        }
    }
    return rtn;
}

Answer 5

Maybe this piece of code helps you:

public void traverse(String path) {
    File root = new File(path);
    File[] list = root.listFiles();
    if (list == null) return;

    for (File file : list) {
        if (file.isDirectory()) {
            traverse(file.getAbsolutePath());
            System.out.println("Directory: " + file.getAbsoluteFile());
        } else {
            System.out.println("File: " + file.getAbsoluteFile());
        }
    }
}

Answer 6

private static List<File> allFiles = new ArrayList<File>();

private static void processFiles(String rootDirectory) {
    File rootDir = new File(rootDirectory);
    if (rootDir.exists()) {
        traverseDirectories(rootDir);
    }
}

private static void traverseDirectories(File file) {
    // add all files and directories to list.
    allFiles.add(file);
    if (file.isDirectory()) {
        File[] fileList = file.listFiles();
        for (File fileHandle : fileList) {
            traverseDirectories(fileHandle);
        }
    } else {
        // call to process file
        System.out.println("Call to process file " + file.getAbsolutePath());
    }
}