StringBuilder out of memory error when working with large strings in java

Question

I went from String test += str; where test grew exponentially large with thousands and thousands of characters. It took 45 minutes to run, probably from creating large strings and deleting garbage. I then staggered the input like this which brought it to 30 seconds.

This seems like the cheap way to do it, but it worked well:

  if (secondDump.length() > 50)
  {
     intermedDump = intermedDump + secondDump;
     secondDump = "";
  }      

  if (intermedDump.length() > 100)
  {
     thirdDump = thirdDump + intermedDump;
     intermedDump = "";
  }
  if (thirdDump.length() > 500)
  {
     fourthDump = fourthDump + thirdDump;
     thirdDump = "";
  }
  if (fourthDump.length() > 1000)
  {
     fifthDump = fifthDump + fourthDump;
     fourthDump = "";
  }
  //with just this and not sixth.  Runtime>>>> : 77343
  if (fifthDump.length() > 5000)
  {
     sixthDump = sixthDump + fifthDump;
     fifthDump = "";
  }
  //with just this.  Runtime>>>> : 35903Runtime>>>> : 33780
  if (sixthDump.length() > 10000)
  {
     fillerDump = fillerDump + sixthDump;
     sixthDump = "";
  }

I then discovered that StringBuilder exists, and I've been trying to use it since, replacing all string operations with it.

The problem is, I keep getting an java.lang.OutOfMemoryError with a java memory heap overflow. I think the string is just too long to store in memory as a StringBuilder object, because it makes about 1/50th of the progress that my previous code did before crashing with an out of memory error. It's only working with maybe under a thousand characters.

Why can a string hold the entire output and this can't come close? Also, if I append text to JTextPane , how much memory does that need? If I dump the StringBuilder contents to JTextpane and keep appending and clearing StringBuilder that doesn't seem to work either.

Here is the existing code. Page is just an object being passed:

protected void concatPlates(page PageA) throws IOException
{
   if (backend.isFirstPage == false)
   {
      frontend.fillOutputPane("\n                                 " +
         "                                               \n", PageA);
      frontend.fillOutputPane("                                 " +
         "                                               \n", PageA);
      frontend.fillOutputPane("                                 " +
         "                                               \n", PageA);
   }
   for (int i = 0; i < PLATELEN-1; i++)
   {

      if (arrLeftCol[i].length() == 0)
      {
         /////////////////////////////////////////////////////////
         /////////////////////////////////////////////////////////
         frontend.fillOutputPane(arrLeftCol[i].append(
           arrRightCol[i]));
      }
     else
     {
        PageA.tempStrBuff = new StringBuilder(arrLeftCol[i].substring(0,40));
        frontend.fillOutputPane(PageA.tempStrBuff.append(arrRightCol[i]));
     }
     arrLeftCol[i].append("");
     arrRightCol[i].append("");

     backend.isFirstPage = false;
  }
}


//this is the frontend class
public static void fillOutputPane(String s, page PageA)
{
   fillOutputPane(PageA.getStrBuf());
}
public static void fillOutputPane(StringBuilder stringBuild)
{
  try
  {
     str.append(stringBuild);
  }
  catch (java.lang.OutOfMemoryError e)
  {
     System.out.println((str.length() * 16) /8);
     //System.out.println(str);
     System.out.println("out of memory error");
     System.exit(0);
  }
}

Here is the error:

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
at java.lang.AbstractStringBuilder.append(Unknown Source)
at java.lang.StringBuilder.append(Unknown Source)
at java.lang.StringBuilder.append(Unknown Source)
at backend.fill(backend.java:603)
at frontend$openL.actionPerformed(frontend.java:191)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
at javax.swing.AbstractButton.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$000(Unknown Source)
at java.awt.EventQueue$1.run(Unknown Source)
at java.awt.EventQueue$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)

I think this is what a stack trace is:

java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Unknown Source)
at frontend.fillOutputPane(frontend.java:385)
at page.concatPlates(page.java:105)
at backend.setPlate(backend.java:77)
at backend.fill(backend.java:257)
at frontend$openL.actionPerformed(frontend.java:191)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
at javax.swing.AbstractButton.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$000(Unknown Source)
at java.awt.EventQueue$1.run(Unknown Source)
at java.awt.EventQueue$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue$2.run(Unknown Source)
at java.awt.EventQueue$2.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)81240560
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)

Answer 1

Apparently your application doen't have enough memory for complete the operation. So you need to specify memory flags to your virtual machine. You can try the following:

 java -Xms256m -Xmx512m YourApp

Where:

• Xms minimun memory allocated by your program at startup (in the example 256 MB)
• Xmx maximun memory allocated by your program if it need more (in the example 512 MB)

Answer 2

Well one of the things that might happen is the fact that in Java, java.lang.String gets special treatment. Strings are immutable, and thus the JVM places each String object in a pool .The role of this pool, amongs others is the fact that is used as a sort of "cache" where if you create multiple String instances having the actual "text" value identical, the same instance will be reused from the pool. This way creating what seems to be a big number of String object instances with the same text inside will infact revert to having very few actual String instances in memory. On the other hand, if you use the same text to initialize multiple StringBuilder instances, those will actually be separate instances (containing the same text), and thus occupying more memory.

On the other hand, concatenating Strings (such as String c = "a"+"b"; ) will in fact create more object instances that if you do it with StribgBuilder (such as new StringBuilder("a").append("b"); ).