Python：AttributeError和网络抓取中的挑战

Question

我正在尝试从“ http://www.landwatch.com/Philippines_land_for_sale/Land ”中抓取数据； 我需要的是地址和价格信息。 我的方法是在python中使用漂亮的汤模块。 当我检查html页面和编码时，我陷入了困境。 可能有些人给我一些提示，以便我继续前进。 基本上，网络检查表明我需要的信息来自div class = clear属性，这是代码：

from lxml import html
import requests
import bs4 as bs
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = 'http://www.landwatch.com/Philippines_land_for_sale/Land'

#Openning up connection, grabbing the page
uClient = uReq(my_url)
page_html = uClient.read()
soup = bs.BeautifulSoup(page_html,'lxml')
g_data = soup.find_all("div",{"class": "clear property left"})
for item in g_data:
  print(item).contents[0]

谢谢，

Answer 1

您快要到了，地址和价格信息在<div class="propName">的<a>元素中，在<div class="clear property left"> ，您可以在g_data内找到更深的g_data ，如下所示：

import requests
from bs4 import BeautifulSoup
my_url = 'http://www.landwatch.com/Philippines_land_for_sale/Land'
link=requests.get(my_url)
soup = BeautifulSoup(link.content, 'lxml')
g_data =soup.find_all('div',class_='clear property left')
for item in g_data:
    address_price_info = item.find("div",{"class":"propName"}).find('a').text
    print(address_price_info )

输出将是：

   Cebu City, Philippines  1185000, PHP
   Tagaytay, Philippines  $116,000
   Quezon City, Philippines  $2,837,000
   Sta Rosa Laguna, Philippines  15500, PHP
   Makati, Philippines  $5,947,826
   Puerto Princesa City, Philippines  $358,813
   Carcar, Philippines  35000000, PHP
   Lipa City, Philippines  $57,750
   Makati, Philippines  6400000, PHP
   Taytay, Philippines  $2,300,000
   Taguig, Philippines  $504,208
   Taguig City, Philippines  $13,760
   Quezon City, Philippines  58000000, PHP
   Cebu City, Philippines  7799030, PHP
   Las Pinas, Philippines  $468,000

更新：

如果您使用Chrome浏览器检查地址和价格信息，它将向您显示位置：

<div class="clear property left">

    <div class="margintop">

        ...             
        <div class="propName"> #Here is the address and price info
            <a href="/Cebu-City-Philippines-Land-for-sale/pid/119211639" onclick="WC('119211639', '-1');"> &nbsp; Cebu City, Philippines <BR/> 1185000, PHP</a>
        </div>


            <div>PAYMENT SCHEMES:\r\rReservation Fee : P20,000 (non refundable)\r\r1. SCHEME 1\rCash - 100% with the following discounts\r* 10% for 7 days payment\r* 8%...&nbsp;</div>

        ...

    </div>
    <div class="clear"></div>
</div>