java.util.Scanner#findWithHorizon throws OutOfMemory exception on 32Mb input stream

Question

Search string '4914904' exists at the tail of the stream.

Here's the code

    Scanner sc = new Scanner(xmlInputStream, "UTF-8");
    if(sc.findWithinHorizon('4914904', 0) != null) { // <--- exception is thrown here
    }

Any suggestions would be highly appreciated.

Answer 1

If you read the API for Scanner, you will see that if you pass the argument 0 to findWithinHorizon that it will read the entire buffer at once.

Since you don't do anything with the value from this I see a few options.

Try changing to useDelimiter(String pattern) and then call if(sc.hasNext()) which may help some with the memory footprint.

If you have XML, use an XML parser instead of a text scanner.

You could consider writing a custom method which parses the input stream one line at a time and perform the search. That way you don't have to read in the full buffer.

Increase the memory you give the jvm when it starts -Xmx256m

On a side note: Don't re-write the code when you post here. Just copy and paste.

Answer 2

Use -Xmx256m when starting Java, it will give the JVM more heap space.

You also might want to consider using an XML parsing library instead of Scanner, if you're dealing with XML. A streaming API like SAX or StAX are the best bet for a big input.