RandomAccessFile filter with read write mode does not work on linux environment

Question

I need help in solving an issue regarding filtering files which have not been completely written as I am getting an exception in that case.

Scenario is: I have a program which takes in files from a queue and puts it in a location on server and another program which reads those files from that location, both these programs runs continuously and the consumer keeps poling to check if there are any files present in the location resulting in reading incomplete xml files at times and throws an error while parisng.

I have written a code to read only those files which can be accessed in read write mode, which seems to work on windows environment but fails on linux.

Code snippet that runs on windows is as below:

listOfFiles = folder.listFiles(new FileFilter() {
   public boolean accept(final File file) {
         try (RandomAccessFile stream = new RandomAccessFile(file, "rw")) {
                 return true;
         } catch(final Exception e) {
              System.out.println("skipping file: file is not completely written");
         }
         return false;
   }
});

Can someone help me with it, why this code is running on windows but not on linux or is there a better solution to this issue?

Answer 1

You are using the wrong technique if you are depending on R/W open telling you if the file is available.

Given all the different types of filesystems available there is no builtin Java mechanism for ensuring that a file is "complete". Since you control both ends of the process, the solution is quite simple.

The process that moves the files to the location where they are to be processed writes the file using a temporary filename format that the reading process knows to ignore. When the writing process completes the writing of a file it renames the file to a name format that the reading process knows is complete. Then, by definition, if a file is "visible" to the reading process it is complete.

The steps would look something like this:

1. The writing process creates a file in the reading location, named TEMP_nnnn_ where nnnn is chosen to make the filename unique. The reading process knows to ignore files whose names start with TEMP.
2. The writing process copies data to the file.
3. When the file is complete the writing process closes the file (to flush all data to disk) and renames it to FILE_nnnn_.
4. The reading process becomes aware of FILE_nnnn_ and knows the file is ready to read.