JUnit tests fail when creating new Files

Question

We have several JUnit tests that rely on creating new files and reading them. However there are issues with the files not being created properly. But this fault comes and goes.

This is the code:

@Test
public void test_3() throws Exception {

    // Deletes files in tmp test dir

    File tempDir = new File(TEST_ROOT, "tmp.dir");
    if (tempDir.exists()) {
        for (File f : tempDir.listFiles()) {
            f.delete();
        }
    } else {
        tempDir.mkdir();
    }

    File file_1 = new File(tempDir, "file1");
    FileWriter out_1 = new FileWriter(file_1);
    out_1.append("# File 1");
    out_1.close();

    File file_2 = new File(tempDir, "file2");
    FileWriter out_2 = new FileWriter(file_2);
    out_2.append("# File 2");
    out_2.close();

    File file_3 = new File(tempDir, "fileXXX");
    FileWriter out_3 = new FileWriter(file_3);
    out_3.append("# File 3");
    out_3.close();
            ....

The fail is that the second file object, file_2, never gets created. Sometimes. Then when we try to write to it a FileNotFoundException is thrown

• If we run only this testcase, everything works fine.
• If we run this testfile with some ~40 testcases, it can both fail and work depending on the current lunar cycle.
• If we run the entire testsuite, consisting of some 10*40 testcases, it always fails.

We have tried

• adding sleeps (5sec) after new File , nothing
• adding while loop until file_2.exists() is true but the loop never stopped
• catching SecurityException, IOException and even throwable when we do the New File(..) , but caught nothing.

At one point we got all files to be created, but file_2 was created before file_1 and a test that checked creation time failed.

We've also tried adding file_1.createNewFile() and it always returns true.

So what is going on? How can we make tests that depend on actual files and always be sure they exist?

This has been tested in both java 1.5 and 1.6, also in Windows 7 and Linux. The only difference that can be observed is that sometimes a similar testcase before fails, and sometimes file_1 isn't created instead

Update

We tried a new variation:

File file_2 = new File(tempDir, "file2");
while (!file_2.canRead()) {
    Thread.sleep(500);
    try {
        file_2.createNewFile();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

This results in alot of Exceptions of the type:

java.io.IOException: Access is denied
 at java.io.WinNTFileSystem.createFileExclusively(Native Method)
 at java.io.File.createNewFile(File.java:883)

... but eventually it works, the file is created.

Answer 1

Are there multiple instances of your program running at once? Check for any extra instances of javaw.exe running. If multiple programs have handles to the same file at once, things can get very wonky very quickly.

Do you have antivirus software or anything else running that could be getting in the way of file creation/deletion, by handle?

Answer 2

Don't hardcode your file names, use random names. It's the only way to abstract yourself from the various external situations that can occur (multiple access to the same file, permissions, file system error, locking problems, etc...).

One thing for sure: using sleep() or retrying is guaranteed to cause weird errors at some point in the future, avoid doing that.

Answer 3

Try File#createTempFile , this at least guarantees you that there are no other files by the same name that would still hold a lock.

Answer 4

I did some googling and based on this lucene bug and this board question seems to indicate that there could be an issue with file locking and other processes using the file.

Since we are running this on ClearCase it seems plausible that ClearCase does some indexing or something similar when the files are being created. Adding loops that repeat until the file is readable solved the issue, so we are going with that. Very ugly solution though.