简体   繁体   English

BeautifulSoup无法抓取Google搜索结果?

[英]BeautifulSoup can't crawl google search results?

Trying to crawl through google search results. 尝试抓取Google搜索结果。 This code works pretty well with all the other sites, I have tried, however not working with google. 我尝试过,此代码在所有其他网站上都可以很好地工作,但是不适用于google。 It returns an empty list. 它返回一个空列表。

from BeautifulSoup import BeautifulSoup
import requests

def googlecrawler(search_term):
    url="https://www.google.co.in/?gfe_rd=cr&ei=UVSeVZazLozC8gfU3oD4DQ&gws_rd=ssl#q="+search_term
    junk_code=requests.get(url)
    ok_code=junk_code.text
    good_code=BeautifulSoup(ok_code)
    best_code=good_code.findAll('h3',{'class':'r'})
    print best_code


googlecrawler("healthkart") 

It should return something like this. 它应该返回这样的内容。

<h3 class="r"><a href="/url?  sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=6&amp;cad=rja&amp;uact=8&amp;ved=0CEIQFjAF&amp;url=http%3A%2F%2Fwww.coupondunia.in%2Fhealthkart&amp;ei=qFmfVc2fFNO0uASti4PwDQ&amp;usg=AFQjCNFHMzqn-rH4Hp-fZK0E4wwxJmevEg&amp;sig2=QgwxMBdbPndyQTSH10dV2Q" onmousedown="return rwt(this,'','','','6','AFQjCNFHMzqn-rH4Hp-fZK0E4wwxJmevEg','QgwxMBdbPndyQTSH10dV2Q','0CEIQFjAF','','',event)" data-href="http://www.coupondunia.in/healthkart">HealthKart Coupons: July 2015 Coupon Codes</a></h3>

Looking at good_code i can't see a h3 or class "r" at all. 看着good_code我根本看不到h3class "r" That would be why your code is returning an empty list. 这就是为什么您的代码返回空列表的原因。

There is no problem with your code as such, but rather, that what you are searching for is not there. 这样的代码就没有问题,而是您所搜索的内容不存在。

What were you expecting to return? 您希望返回什么?

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM