Java：一次按“ n”条记录读取大文件

Question

I have an input text file having 5k records. I want to read in 1k records, pass as parameter to SQL query IN clause (which is limited to 1k values) and then get the next 1k records from text file and repeat. How can I do this in java. Below is sample code i have so far:

FileInputStream fis = null;
    BufferedReader reader = null;

    try {
        fis = new FileInputStream("values.txt");
        reader = new BufferedReader(new InputStreamReader(fis))
        int i,counter=0;
        do{
        String line;
        ArrayList values = new ArrayList();
        for(i=0;i<1000;i++){
            line = reader.readLine();
            values.append(line);
            if(line ==null)
                break;
        }
        counter = i;
        String sql = ....;
        //db connection
        }while(counter<6000);
        conn.close;
     }

Answer 1

You can improve your reading speed using buffers to be able to read a big chunk in every single file access. So specify the buffer size of your BufferedReader like :

BufferedReader br = new BufferedReader(new FileReader("values.txt"), 1000);

or use a buffer of your own to read whole file to memory like:

char[] buffer = new char[1000];
Reader r = new FileReader("values.txt");
int read;
do {
    read = r.read(buffer);
    // store buffer in a StringBuilder
} while (read > 0);
//read StringBuider using a buffered reader

It depends on your requirements and how big is the file

Answer 2

You can create a BufferedReader of any size for example the bottom style creates a 1 KB buffered reader. Probably you should look around how many lines of your file makes 1 KB or more and configure it accordingly.

BufferedReader br;
br = new BufferedReader(new InputStreamReader(new FileInputStream("values.txt"), "UTF-16"), 1*1024);

You can read the documentation of BufferedReader