BigQuery - 如何使用 Java API 迭代结果

Question

I am trying to download a query result of 90k rows (this amount states the web UI and the resulting object) but I obtain more than 130k items in the iterator as follows:

QueryRequest queryRequest = QueryRequest
                                        .newBuilder("......")
                                        .setUseLegacySql(true)
                                        .build();
QueryResponse response = bigquery.query(queryRequest);
QueryResult result = response.getResult();
Integer c = 0;

while(result != null){
          Iterator<List<FieldValue>> iter = result.iterateAll();
          while(iter.hasNext()){
              iter.next();
              c++;
            }    
          result = result.getNextPage();
}

At the end of the reading is about 130K but reuslt.getTotalRows() contains 90K.

Do you have any idea on what I'am doing not correctly?

Answer 1

From JavaDoc , Page.iterateAll() returns results from all pages.

You get all results (from the first page), then advance to the next page, where you get all results again (minus the first page). Setting the page size to a smaller number actually increases the results you get.

Your code should look like this:

QueryRequest queryRequest = QueryRequest
                                    .newBuilder("......")
                                    .setUseLegacySql(true)
                                    .build();
QueryResponse response = bigquery.query(queryRequest);
QueryResult result = response.getResult();
Integer c = 0;

Iterator<List<FieldValue>> iter = result.iterateAll();
while(iter.hasNext()){
    iter.next();
    c++;
}