Too many open files in Java

Question

i'm call this method several times:

  private static void writeFile(double val, String myFile) throws IOException { FileWriter file = new FileWriter(myFile, true); file.write(val + "\\n"); file.close(); } 

After a periode i have this exception :

 Exception in thread "main" java.io.FileNotFoundException: myFile (Too many open files) 

How i can i resolve this exception please ?

Answer 1

I'm guessing that some (maybe all) of your file writes are resulting in exceptions which is short circuiting the file.close(). Even if that is not the case, it is common practice to wrap your File IO operations in try/catch/finally to ensure that anything you open gets closed regardless of whether the read/write succeeded or not. If you don't close the files, you will run out of handles.

private static void writeFile(double val, String myFile) throws IOException {
    FileWriter file = null;
    try {
        file = new FileWriter(myFile, true);
        file.write(val + "\n");
    } catch( IOException e ) {
        e.printStackTrace();
    } finally {
        if( file != null ) {
            file.close();
        }
    }
}

or, if using JDK 1.7:

private static void writeFile(double val, String myFile) throws IOException {
    try (FileWriter file = new FileWriter(myFile, true)) {
        file.write(val + "\n");
    } catch( IOException e ) {
        e.printStackTrace();
    }
}

Answer 2

As mentioned, the close probably is not done, due to some IOException, so in Java 7:

private static void writeFile(double val, String myFile) throws IOException {
    try (FileWriter file = new FileWriter(myFile, true)) {
        file.write(val + "\n");
    }
}

The question arises: does the code hide exceptions?

The real cause however, is that you probably are writing to the same file, at more or less the same time. For this kind of work, loggers are more suited.

With java.util.logging:

private static void writeFile(double val, String myFile) throws IOException {
    Logger.getLogger(MyClass.class.getName()).log(Level.INFO, "" + val);
}

In logging.properties you then can use a file logger for MyClass, with an as-is output format.

Answer 3

I am writing to 1000000 files on my machine and I don't get any error.

import java.io.FileWriter;
import java.io.IOException;

public class ManyFiles {
    private static void writeFile(double val, String myFile) throws IOException {
        FileWriter file = new FileWriter(myFile, true);
        file.write(val + "\n");
        file.close();
    }

    public static void main(String[] args) {
        for (int i = 0; i < 1000000; i++) {
            try {
                writeFile(i + 1.0, "/tmp/scratch/" + i);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

For me all the files are generated, with no failures:

$ ls /tmp/scratch | wc -l
1000000

Are you spawning multiple threads?

You must explain more about your environment:

• Which Operating System are you using ?
• Which JDK version you are using?
• What is the hardware configuration?
• Are there any system limits to the user you are running this program as?

My system configuration is below:

$ uname -a
Linux box01 3.11.8-300.fc20.x86_64 #1 SMP Wed Nov 13 16:34:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
$ javac -version
javac 1.7.0_45
$ java -version
java version "1.7.0_45"
OpenJDK Runtime Environment (fedora-2.4.3.0.fc20-x86_64 u45-b15)
OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)
$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 62970
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Answer 4

A crude idea is to do a System.out.println() or write to the log file each time you open a file, and each time you close it. When it crashes, look at the log for opens without closes.
Another consideration is if you are calling the function faster than it can perform its open/close on the file. In that case, a synchronized block may be useful for your function to prevent concurrent access.

Answer 5

Use Java 7 new feature Try-with-resource whenever you are intending to use any external resource and open the resource to start using.
This new feature excessively helpful with that thought you forgot to close a resource after open, it will automatically close that resource after use.