用Python做一个简单的Yahoo搜索

Question

I need to write a Python script which, at one point, does a Yahoo web search to find and download a bunch of C source files. I'm very new to this and I can't figure out how to just get started with doing a simple web search... I've seen a lot of stuff about BOSS but, from my understanding, this is something you need to pay to use? I am not willing to pay for this.

I've used Python YQL to get some RSS results as follows:

import yql
y = yql.Public()
result = y.execute('select * from rss where url="http://www.un.org/apps/news/rss/rss_top.asp"');

for row in result.rows:
   print row.get('title')

And this seems to work, but I can't figure out how to just do a normal web search (since the search.web table is apparently gone). A basic working example would be much appreciated.

Answer 1

You might want to try it using mechanize , which simulates a browser. If you need to clean out some of the crud in the resulting files, use Beautiful Soup.

Answer 2

I can't figure out how to just do a normal web search (since the search.web table is apparently gone). A basic working example would be much appreciated.

You can use Bing as your search provider and use the microsoft.bing.web data table to perform a web search.

A basic example in Python, which prints the titles of the first 10 results for cake , might look like:

import yql
y = yql.Public()
env = "http://datatables.org/alltables.env"
query = "select * from microsoft.bing.web where query=@query"

results = y.execute(query, {"query": "cake"}, env=env)

for row in results.rows:
    print row.get("Title")