java.lang.OutOfMemoryError：

Question

I'm trying to create video files from the bytes retrieved from the database. The program was worked well before few hours. After uploading a large file, When I try to retrieve it, It is producing the error java.lang.OutOfMemoryError:

My code is:

    conn = prepareConnection();
    StringBuilder sb=new StringBuilder(1024);
    sb.append("select videoname,videoid,videofull from ").append(uname.trim()).append("video");
    String sql=sb.toString();

    stmt = conn.prepareStatement(sql);
    ResultSet rs = stmt.executeQuery();

    while(rs.next()){
         byte[] videoData = rs.getBytes("videofull");       //#57
         int vid=rs.getInt("videoid");
         StringBuilder sb1 = new StringBuilder();
         sb1.append(vid);
         String videoid=sb1.toString();
         String vname=rs.getString("videoname");
         File file=new File("C:/Users/JamesPJ/Documents/skypark/skypark/WebContent/sp/resources/videos/"+vname+""+videoid+".mp4");

        if(file.exists() && !file.isDirectory()){
            continue;
        }
        else
        {
         FileOutputStream output = new FileOutputStream(file);
         IOUtils.write(videoData, output);
         output.close();
        }
       }
    request.setAttribute("uname", uname);
    RequestDispatcher dispatcher = request.getRequestDispatcher("/VideoList");
    if(dispatcher != null) {
        dispatcher.forward(request, response);
    } 

Console output is:

Exception in thread "http-bio-8080-exec-3" java.lang.OutOfMemoryError: Java heap space
at oracle.sql.BLOB.getBytes(BLOB.java:217)
at oracle.jdbc.driver.T4CBlobAccessor.getBytes(T4CBlobAccessor.java:462)
at oracle.jdbc.driver.OracleResultSetImpl.getBytes(OracleResultSetImpl.java:716)
at oracle.jdbc.driver.OracleResultSet.getBytes(OracleResultSet.java:402)
at skypark.VideoFileCreator.doGet(VideoFileCreator.java:57)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:412)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339)
at skypark.VideoStream.processRequest(VideoStream.java:48)
at skypark.VideoStream.doGet(VideoStream.java:64)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:931)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

Please anyone tell me what this error tells. What I need to correct in it.......Thanks.....

Answer 1

Here,

byte[] videoData = rs.getBytes("videofull");

You're storing the entire file contents in server's memory. You know, one byte of a byte[] eats one byte of Java's memory. If you've eg only 500MB memory and the file is over 500MB, then you'll get exactly this error. Note that you will also get it when 10 users concurrently request a video file of 50MB. So increasing the Java memory as suggested by other answers is only a temporary solution and not really well thought out.

You need to obtain it in flavor of an InputStream instead so that would internally allocate only a few (kilo)bytes as streaming buffer in memory instead of the entire file contents.

InputStream videoData = rs.getBinaryStream("videofull");

And write it to the desired output stream using IOUtils#copy() instead (don't forget to close them in the finally!).

FileOutputStream output = new FileOutputStream(file);
try {
    IOUtils.copy(videoData, output);
} finally {
    IOUtils.closeQuietly(output);
    IOUtils.closeQuietly(videoData);
}

Answer 2

As the size of your video files is large it has finished all the memory in heap. You got to increase your heap size by setting the VM arguments as -Xmx1024m . this will increase your heap space to 1 gb . If the problem still persist then you have to use java vm visual to analyse which part of your program is consuming more memory and u shud work on other ways to reduce it. Increasing the Heap space more than 1 gb is not a good solution to over come heap space issue .

If your running your program from an IDE like eclipse set ur configuration in run as configuration mode.

Answer 3

The error says you are out of memory. You need to load less stuff into memory, or increase the total amount of memory available.

Answer 4

The error tells:

OutOfMemoryError: Java heap space

You need more heap space: you can configure the VM to use more heap Space.

Option -Xmx

Example: -Xmx256m for 256 Megabytes maximum heap space

Answer 5

Use the -Xmx flag to increase the heap space allocated to your JVM. -Xmx1500m for example.

If executeing from a cli this is easy

java -Xmx2000m youMainJavaFile

If you are executing from eclipse or the like, youll need to go into the options and tell it to add this flag when execing the JVM.

Answer 6

Thrown when the Java Virtual Machine cannot allocate an object because it is out of memory, and no more memory could be made available by the garbage collector .

Says Oracle docs, this means you should increase the memory limitation of your JVM which runs your program. Increasing heap size:

JVM_ARGS="-Xmx1024m"

This sets the heap size to 1024mb. Or you can do this in your IDE in run configruations giving -Xmx1024m as VM parameter will work properly.

Answer 7

Inside IOUtils.write use output.flush(), so that the buffered bytes will be written to destination and buffer get cleared up. Nothing stays in memory.

Answer 8

StringBuilder sb1 = new StringBuilder();
     sb1.append(vid);
     String videoid=sb1.toString();

That is one way to do it, heres another way :

     String videoId = new String(new Integer(rs.getInt("videoid")));

Also, learn what/how the java heap space is allocated :

    -Xms<size>        set initial Java heap size
    -Xmx<size>        set maximum Java heap size
    -Xss<size>        set java thread stack size

To set minimum heap to 64 MB and maximum heap 256 MB for a Java program HelloWorld : java -Xms64m -Xmx256m HelloWorld If you are using an IDE you will have to change the params in your server startup. HTH

Answer 9

Your heap is too small to hold all the objects allocated by your application.

The easiest solution is to increase the heap size . Eg -Xms1g -Xmx1g

Or, you could try to analyze the app to see where is the memory allocation coming from and optimize that code path.