如何从Google Custom Search API获得100多个结果

Question

I am trying to use Google Custom Search API for research purposes in Java. As a result I'm gonna need a big result set for each query. However it seems that I'm limited with first 100 results which is much less than what I need. I use the list method like this:

list.setStart(90L);

And when I set it to start from index 100 I get this error:

com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request

{   "code" : 400,   
    "errors" : [ {
    "domain" : "global",
    "message" : "Invalid Value",
    "reason" : "invalid"   
  } ],   
    "message" : "Invalid Value" 
}

Is there any way to remove this limitation?

Answer 1

There's a difference between two limitations: a) max 100 queries per day; and b) max 100 results per searched phrase (even split into 10 queries of 10 results per query). The limitation (a) can be solved by paying to Google, whereas limitation (b) cannot be solved at all, it seems to be a feature of the CSE product when searching the whole web (ie not just a site search, which can give more results), see eg https://productforums.google.com/forum/#!topic/customsearch/VM8_6trCxGU

Answer 2

You can only request 10 results per query with Google Custom Search, so you'd want to split up that search into multiple queries. (I don't know how your actual query code works so I don't know if you already know this)

Google Custom Search (Free) also has a limit of 100 queries per day, so if you want more, you'd have to pay.

You can find the pricing here: https://developers.google.com/custom-search/json-api/v1/overview

Answer 3

You can use lowRange and highRange to solve this problem. See: https://productforums.google.com/forum/#!topic/customsearch/2qilVDaCz0A