Java - How to join many list values into a single string with delimiter at end of each value

Question

How to join list of millions of values into a single String by appending '\\n' at end of each line -

Input data is in a List:

   list[0] = And the good south wind still blew behind,
   list[1] =  But no sweet bird did follow,
   list[2] =  Nor any day for food or play
   list[3] =  Came to the mariners' hollo!

Below code joins the list into a string by appending new line character at the end -

String joinedStr = list.collect(Collectors.joining("\n", "{", "}"));    

But, the problem is if the list has millions of data the joining fails. My guess is String object couldn't handle millions line due to large size.

Please give suggestion.

Answer 1

The problem with trying to compose a gigantic string is that you have to keep the entire thing in memory before you do anything further with it.

If the string is too big to fit in memory, you have only two options:

1. increase the available memory, or
2. avoid keeping a huge string in memory in the first place

This string is presumably destined for some further processing - maybe it's being written to a blob in a database, or maybe it is the body of an HTTP response. It's not being constructed just for fun.

It is probably much more preferable to write to some kind of stream (maybe an implementation of OutputStream ) that can be read one character at a time. The consumer can optionally buffer based on the delimiter if they are aware of the context of what you're sending, or they can wait until they have the entire thing.

Preferably you would use something which supports back pressure so that you can pause writing if the consumer is too slow.

Exactly how this looks will depend on what you're trying to accomplish.

Answer 2

Maybe you can do it with a StringBuilder, which is designed specifically for handling large Strings. Here's how I'd do it:

StringBuilder sb = new StringBuilder();
for (String s : list) sb.append(s).append("\n");

return s.toString();

Haven't tested this code though, but it should work