Google API Custom Search with Python - Programmatic Search Results

Question

I am trying to use the Google API Custom Search, and I don't have any clue where to start. It seems you have to create a "custom search engine" in order to parse search results, and you are limited to 100 per day.

What module should I use for this? I believe I start here: http://code.google.com/p/google-api-python-client/

I need an API key or something? Basically I want to be able to do this operation, and Google's documentation is confusing, or perhaps beyond my level.

Pseudocode:

from AwesomeGoogleModule import GoogleSearch

search = GoogleSearch("search term")

results = search.SearchResultsNumber

print results

Basically, that number you get of total results for a particular search term? I want to scrape it. I don't want to go via the front-end Google, because that's very easy to get blocked. I don't need to go beyond the 100 searches that the API allows. This will only be for 30-50 search terms, maybe 80-100 at MOST.

Answer 1

Sample code for Custom Search using the google-api-python-client library is here:

http://code.google.com/p/google-api-python-client/source/browse/#hg%2Fsamples%2Fcustomsearch

You will need to create your own API Key by visiting:

https://code.google.com/apis/console/

Create a project in the APIs Console, making sure to turn on the Custom Search API for that project, and then you will find the API Key at the bottom of the API Access tab.