Reading current and new files from a directory using Java

Question

I have written a program to process files in a directory. At start up it reads the current files in a directory, and then it uses a monitor to discover new files. Once it has processed a file,the program deletes the file. The problem is that there is a time gap, no matter how slight, between reading the files in a directory at startup and then starting the listener. A file created in that gap would be missed. One possible solution would be to repeatedly read the files in a directory (newDirectoryStream), but that doesn't seem as elegant or possibly efficient as using a monitor. The code uses the Apache Commons monitor and looks something like:

// Read Current files
stream = Files.newDirectoryStream(listenDir);
processFile(file);

// Process New files
FileAlterationObserver observer = new 
     FileAlterationObserver(listenDir.toAbsolutePath().toString(),filter);

    
FileAlterationMonitor monitor = new FileAlterationMonitor(POLL_INTERVAL);
FileAlterationListener listener = new FileAlterationListenerAdaptor() {
    @Override
    public void onFileCreate(File file) {
        processFile( file.toPath());
    } 
};
        
observer.addListener(listener);
monitor.addObserver(observer);
monitor.start();
        

Answer 1

Simply flip it: First set up the listener and then obtain a directory stream. Go through a concurrent set which lets you do a 'only once ever' layout (the one that added the file name to the set and got the return value indicating 'you actually are the one that added it, you're not merely re-applying something that was already in there' - then you handle the file, otherwise you keep going). This way, if the file is added right in the 'sweet spot', both the dirstream and the observer would get it, but still only one will process it.