简体   繁体   中英

MemoryMapped File reading from a huge file separated by new line character

I am reading a huge file using memory mapped I/O. The problem which i came across is, i am reading MemoryMappedByteBuffer character by character the . So i need to pass multiple strings present in the file which are separated by "\\n".

        RandomAccessFile aFile = new RandomAccessFile(this.getFileName(), "r");
        FileChannel inChannel = aFile.getChannel();
        MappedByteBuffer buffer = inChannel.map(FileChannel.MapMode.READ_ONLY, 
        0, inChannel.size());

        buffer.load(); 
        for (int i = 0; i < buffer.limit(); i++)
        {   // There are many strings in the file separated by \n
            System.out.println((char) buffer.get() == '\n'); // Gives true 
             //need to make a complete string over here.
        }
        buffer.clear(); // do something with the data and clear/compact it.           
        return null; // The String which has been made in the above for loop

This is not the answer you're asking for, as this does not use a memory-mapped file. If it's truly not helpful, I'll remove it.

I am reading a huge file using memory mapped I/O

If the file really is huge, then the approach you're taking is going to be quite memory demanding.

An alternative is to use a BufferedReader , and this makes your task quite trivial:

final List<String> lines = new ArrayList<>();
final BufferedReader br = new BufferedReader(
    new InputStreamReader(new FileInputStream(file), charset));

String line;
while ((line = br.readLine()) != null)
{
  lines.add(line);
}

return lines;

An equivalent of this code is already included in the JDK utility method Files.readAllLines(Path,Charset) :

Read all lines from a file. This method ensures that the file is closed when all bytes have been read or an I/O error, or other runtime exception, is thrown. Bytes from the file are decoded into characters using the specified charset.

 public static List<String> readAllLines(Path path, Charset cs) throws IOException 

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM