Best way to use RandomAccessFile is Java

Question

I am creating a utility which write MSSQL table blob to data disk file using RandomAccessFile. It is too slower because we need to seek the last position always and write the stream contents.. please let me know any other alternative to speedup the randomaccessfile writing.

I have more than 50M records, with current logic it took approx 10 hours.

my code block is something like that:

RandomAccessFile randomAccessFile = new RandomAccessFile(file, "rw");
InputStream inputStream = null;

while (rows.hasNext()) {
    Row row = rows.next();
    inputStream = (InputStream) row.getValues()[0];
    offset = randomAccessFile.length();
    byte[] buffer = new byte[8196];
    int count;
    randomAccessFile.seek(offset);
    randomAccessFile.setLength(offset);
    while ((count = inputStream.read(buffer)) != -1) {
        randomAccessFile.write(buffer, 0, count);
    }
}
randomAccessFile.close();   

Answer 1

According to the code you posted, you only need to append to an existing file. This is done easier and more efficient using a buffered writer in append mode.

Thus, use

BufferedWriter writer = Files.newBufferedWriter(file.toPath(), StandardOpenOptions.CREATE, StandardOpenOptions.APPEND);

instead.

Update after Peter's comment: for an output stream, the whole thing is basically the same, only that Files does not have a nice convenience function for the "buffered" part. Therfore:

OutputStream outputStream = new BufferedOutputStream(Files.newOutputStream(file.toPath(), StandardOpenOption.CREATE, StandardOpenOption.APPEND));

Answer 2

Currently you are writing roughly 8 Kb (8196 / 1024) of data in each iteration and each iteration does an I/O operation which is blocking and takes time. Try to increase this to at least 1 Mb roughly (10,000,000).

byte[] buffer = new byte[10000000];