How to read Big String Chunks Wise?

Question

I've a below set of strings in a file for a sample

ABCBDJHJHD#NASNAJBSJBSJBSBS#JAJBAJBSBSBSBS#AHBAHHSBSBSBVSVBVS#HGVGFGFGF
#JKHGHGHG#JHJHJHBHBHHGG#
HGFGFGJVVGV#JHBHBHBHB

The size of one record can in GBs too.!

Inside string, # is a separator. So Is there any way I could read line in small packets or smallest entity like bits so that I dont get memory issues ?

I just want to keep reading # separated values from disk rather than putting whole big line in Memory and take then further for my processing.

Any suggestions please...!

Thanks

Answer 1

Sure - that's just like what BufferedReader.readLine does with the line separator as the separator. ( readLine() is more complicated because \\r\\n and \\n are both separators)

public static String readUntil(BufferedReader r, char separator) throws IOException {
    StringBuilder b = new StringBuilder();
    int ch;
    while ((ch = r.read()) != -1) {
        if (ch == separator) {
            return b.toString();
        } else {
            b.append((char) ch);
        }
    }
    if (b.length() == 0) {
        return null;
    } else {
        return b.toString();
    }
}

and invoke it like this:

BufferedReader r = new BufferedReader(new FileReader(file));
String nextString;
while ((nextString = readUntil(r, '#')) != null) {
    // Do something with nextString
}