How to add string to a file without closing the output stream objects every time

Question

I will receive a large chunk of data (say 1000 data/second, each data has minimum size of 15 bytes). My earlier approach was to create a new outputstream object every time, specify the path and add the values to the file, all this is done in a separate thread. How ever I am still facing a performance hit. I taught instead of writing the data to a file as

File dir = new 
File(android.os.Environment.getExternalStorageDirectory().getAbsolutePath() + DEBUG_FILE_PATH);
boolean b = dir.mkdirs();
try
{
    fileOutputStream = new FileOutputStream(new File(dir, FILE_NAME), 
    true);
    outputStreamWriter = new OutputStreamWriter(fileOutputStream);
}
catch (FileNotFoundException e)
{
    e.printStackTrace();
}

outputstreamwriter.append("some data").close(); 

I want to maintain the outputstreamwriter and other objects and use them to add the data to the outputstreamwriter buffer, and at the end of my application(when i close the app, may at onDestroy() method of activity). I need to write the data to the file and then close all the open stream.

This approach works for me, but the buffer size for outputstreamwriter is 8kb only. Which is less compared to the amount of data that I am receiving.

How can i solve this ?

Answer 1

The vast majority of your performance hit is most probably in opening the file every single time you want to write a few bytes to it.

So, if you just eliminate the opening and closing of the file all the time, you should be fine.

Just open the file once, keep writing data to it as the data arrives, and then close the file when your application closes.

This way, using a buffered OutputStreamWriter will give you a performance benefit without having to worry about the size of its buffer: when its buffer is full, it will flush itself, transparently to you. You don't need to know anything about how it works and how large (or small) its buffer is.

Answer 2

This solved my problem. By this approach I have opened the file once when the app starts and, I am adding the 'n' values which I receive from the service into the file. I am flushing the buffer(this writes the data to file). Now even if I receive large data not more than 8kb(buffer max size) I can write it to file which is already opened. Finally I am closing the streams when the app is closed.

//Util class
public static File dir;
public static FileOutputStream fileOutputStream;
public static OutputStreamWriter outputStreamWriter;

//Different class, you can initialize it on your Application class or home activity
private void initializeFileWriteObjects()
{
    dir = new File(android.os.Environment.getExternalStorageDirectory().getAbsolutePath() + DEBUG_FILE_PATH);
    boolean b = dir.mkdirs();
    try
    {
        fileOutputStream = new FileOutputStream(new File(dir, FILE_NAME), true);
    }
    catch (FileNotFoundException e)
    {
        e.printStackTrace();
    }
    outputStreamWriter = new OutputStreamWriter(fileOutputStream);
}

//Util File
private static boolean writeToFile(final byte[] stream)
{
    //Convert String to hex, as I want this to be in hex
    final String stringData = byteArrayToHexString(stream);

    try
    {
        outputStreamWriter.append(stringData);
        outputStreamWriter.flush();
    }
    catch (IOException e)
    {
        e.printStackTrace();
        return false;
    }
    return true;
}

//When the app is closed.
@Override
protected void onDestroy()
{
    super.onDestroy();
    closeFileStream();
}

//This method is in same Util class, but called at onDestroy()
public static void closeFileStream()
{
    try
    {
        outputStreamWriter.close();
        fileOutputStream.close();
    }
    catch (IOException e)
    {
        e.printStackTrace();
    }
}