How to convert List to the csv byte array safely?

Question

Initially I had the following code:

Attempt 1

try (var output = new ByteArrayOutputStream();
     var printer = new CSVPrinter(new OutputStreamWriter(output), CSVFormat.DEFAULT)) {
   printer.printRecord(EMAIL);
   for (MyBean mb : items) {
     printer.printRecord(mb.getEmail());
   }
   externalHttpCall(output.toByteArray());
}

Here I found out that sometimes the byte array is not written fully.

I understand that it is because of the fact that stream is not flushed during externalHttpCall invocations.

To fix it I wrote the following:

Attempt 2

try (var output = new ByteArrayOutputStream();
     var printer = new CSVPrinter(new OutputStreamWriter(output), CSVFormat.DEFAULT)) {
  printer.printRecord(EMAIL);
  for (MyBean mb : items) {
    printer.printRecord(mb.getEmail());
  }
  printer.flush();
  log.info("Printer was flushed");

  externalHttpCall(output.toByteArray());
}

It solved the problem, but here I was lost in a thought that it is really bad idea to close stream only after externalHttpCall . So I came up with the following solution:

Attempt 3

externalHttpCall(convertToByteArray(items);

public byte[] convertToByteArray(List<MyBean> items){
  try (var output = new ByteArrayOutputStream();
       var printer = new CSVPrinter(new OutputStreamWriter(output), CSVFormat.DEFAULT)) {
    printer.printRecord(EMAIL);
    for (MyBean mb : items) {
      printer.printRecord(mb.getEmail());
    }
    return output.toByteArray();    
  }
}

I expected that flush will happen before stream close. But based on my experiments it doesn't work. Looks like it happens because of flush happens before stream close but after toByteArray invocation.

How could I fix it?

Answer 1

Given the three code snippets in the question I'd assume that this should work:

externalHttpCall(convertToByteArray(items);

public byte[] convertToByteArray(List<MyBean> items){
  try (var output = new ByteArrayOutputStream();
       var printer = new CSVPrinter(new OutputStreamWriter(output), CSVFormat.DEFAULT)) {
    printer.printRecord(EMAIL);
    for (MyBean mb : items) {
      printer.printRecord(mb.getEmail());
    }
    printer.flush()
    log.info("Printer was flushed");

    return output.toByteArray();
  }
}

Depending on the CSVFormat the CSVPrinter is flushed automatically on close ( CSVFormat.DEFAULT will not be flushed automatically...). You can use CSVFormat 's builder like pattern to make the format flush on close with CSVFormat.DEFAULT.withAutoFlush(true) (thanks to @PetrBodnár for this hint). This will however probably make no difference in the above example.

If you translate the try-with-resource to the actual call order you will get something like this:

var output = new ByteArrayOutputStream();
var printer = new CSVPrinter(new OutputStreamWriter(output), CSVFormat.DEFAULT)
printer.printRecord(EMAIL);
...
var result = output.toByteArray();
printer.close();  // might call flush
output.close();
return result;

As the close operations will be called in the finally-block, they will take place after creation of the byte array. If flush is needed, you will need to do it prior to calling toByteArray .

Answer 2

The following is a correct usage:

var output = new ByteArrayOutputStream();
try (var printer = new CSVPrinter(
            new OutputStreamWriter(output, StandardCharsets.UTF_8), CSVFormat.DEFAULT)) {
    printer.printRecord(EMAIL);
    for (MyBean mb : items) {
        printer.printRecord(mb.getEmail());
    }
}
// Everything flushed and closed.
externalHttpCall(output.toByteArray());

This error behavior might stem from something else.

For instance the externalHttpCall not flushing . Or writing the bytes as text (using a Writer io OutputStream), and expecting UTF-8, whose multi-byte sequences are brittle, maybe raising an exception. Or setting the HTTP header Content-Length wrong, as String.length().

An other cause: items containing a null or getEmail throwing an exception that is not detected.

Available is also:

String s = output.toString(StandardCharsets.UTF_8);