Write string to huge file

Question

I know there are several threads about this problem but i think my problem is a little bit different because of the size.

In my example I want to write 1,7 million lines to an text file. In worst case there could be much more. This lines are create for the sql loader to load fast data into a table so the file could be very large because sql loader could handle that.

Now I want to write the big file as fast as I could. This is my actually method:

BufferedWriter bw = new BufferedWriter(new FileWriter("out.txt"),40000);
int u=profils.size()-1;
for(int z=0; z<u;z++){
  for(int b=0;b<z;b++){
    p = getValue();
    if(!Double.isNaN(p) & p > 0.55){
      bw.write(map.get(z) + ";" + map.get(b) + ";" + p + "\n");
    }
  }
}
bw.close();

For my 1,7 million lines I need about 20 minutes. Can I handle that faster with any method that I don't know?

FileChannel:

File out = new File("out.txt");
FileOutputStream fileOutputStream = new FileOutputStream(out, true);
FileChannel fileChannel = fileOutputStream.getChannel();
ByteBuffer byteBuffer = null;

int u=profils.size()-1;
for(int z=0; z<u;z++){
  for(int b=0;b<z;b++){
   p = getValue();
     if(!Double.isNaN(p) & p > 0.55){
        String str = indexToSubstID.get(z) + ";" + indexToSubstID.get(b) + ";" + p + "\n";
        byteBuffer = ByteBuffer.wrap(str.getBytes(Charset.forName("ISO-8859-1")));
        fileChannel.write(byteBuffer);
      }
   }
 }
fileOutputStream.close();

Answer 1

FileChannel is your way to go. It is used for huge amount of writes. Read the api documentation here