Java仅从大文件逐行读取\\ n

Question

I have a file that records terminated by "\\n" and columns terminated by X"01", the first non printing character. And it is big... 7GB which will totally screw my laptop memory.

I have done some google around how to read big file line by line by using BufferReader .. etc. But the definition of LINE is a bit different, the readline function will return the line that either ends with "\\n", "^M" ..etc.

I am wondering is there a solution in Java 6/7 to read big files line by line, whose definition is the line end with \\n ONLY.

Thanks!

I have a sample data set here and wondering if some one who could run against the sample data and extract the first column timestamp of every line.

here is what I have done but it only reads in the first line,

import java.io.File;
import java.io.IOException;
import java.util.Scanner;

public class ParseAdafruit {

    public static void main(String[] args) throws IOException {
        // Predefine the delimiter ^A
        String delimiter = String.valueOf((char) 1);

        Scanner scanner = new Scanner(new File("/Users/.../data")).useDelimiter("\\n");
        while (scanner.hasNext()) {
            String line = scanner.next(); // This is your line
            String[] parts = line.split(delimiter);
            System.out.println(parts[0]);
        }
    }
}

Output

2014-01-28 18:00:41.960205

btw, I had such a good time in Python by using something like this:

for line in sys.stdin: 
    print line.split(chr(1))[0]

Answer 1

This is how to set a Scanner to separate the string in a file by "\\n". I don't know what you do with each line, but if you want to read the file into a string use a StringBuilder (or StringBuffer for synchronization) because String is immutable.

Scanner scanner = new Scanner(new File("PathToFile")).useDelimiter("\\n");
while (scanner.hasNext()) {
    scanner.next(); // This is your line
}

Answer 2

it seems that the file encoding matters, so we read in the file as UTF-8 before running the scanner

import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStreamReader;
import java.util.Scanner;

...

String fileDir = "pathtodata";
try
{
    BufferedReader in = new BufferedReader(new InputStreamReader(
            new FileInputStream(fileDir), "UTF8"));

    Scanner scanner = new Scanner(in).useDelimiter("\\n");
    while (scanner.hasNext())
    {
        String line = scanner.next(); // This is your line
        String[] parts = line.split(delimiter);
        System.out.println(parts[0]);
    }
    scanner.close();
    in.close();
}
catch (Exception e)
{
    e.printStackTrace();
}