在Beautiful Soup或Selenium的標簽中獲取數據

Question

我正在嘗試從該網站的<b>標記中提取內容

我想通過輸入地址來提取不同城市的內容。

Query Date: Wed Aug 09 2017
Latitude: 33.4484
Longitude: -112.0740

ASCE 7-10 Windspeeds 
(3-sec peak gust in mph*):

Risk Category I: 105
Risk Category II: 115
Risk Category III-IV: 120
MRI** 10-Year: 76
MRI** 25-Year: 84
MRI** 50-Year: 90
MRI** 100-Year: 96

ASCE 7-05 Windspeed:
  90 (3-sec peak gust in mph)
ASCE 7-93 Windspeed:
  72 (fastest mile in mph)

下面給出了我嘗試過的代碼。

from bs4 import BeautifulSoup
from datetime import datetime
import dateutil.parser
import urllib2
import requests
import sys
import re
import csv
import pandas as pd
from selenium import webdriver

chrome_path = r"/usr/local/share/chromedriver"
driver = webdriver.Chrome(chrome_path)
driver.get("http://windspeed.atcouncil.org/") # opening the site
driver.find_element_by_xpath(
"""//*[@id="address"]""").click() # click the radio button
driver.find_element_by_xpath("""//*[@id="google-map-address"]""").click() # clicking the textbox
cities = ['pheonix'] # city list
for city in cities:
# print (city)
    driver.find_element_by_xpath("""//*[@id="google-map-address"]""").send_keys(city) # passing cities
    driver.find_element_by_xpath("""//*[@id="searchform"]/div[1]/div[2]/button""").click()
    driver.find_element_by_xpath("""// *[ @ id = "latt"]""")
    driver.find_element_by_xpath('//*[@id="searchform"]/div[1]/div[7]/span/input').click()
    x = driver.current_url
print x


Data = {'optionCoordinate': '2','coordinate_address': cities}
page = requests.post(x, data = Data)
soup = BeautifulSoup(page.content,'html.parser') 
for b_tag in soup.find_all('b'):
    print b_tag.text,b_tag.next_sibling

如果可以使用Selenium和Python BS4完成，請幫助我找到解決方案。

Answer 1

您可以簡單地使用硒來提取這些數據：

from selenium import webdriver as wd

br = wd.Chrome()
br.get(URL)  # use url mentioned in question
s = br.find_element_by_id('bodyContent')  #search results div
print '\n'.join(s.text.split('\n')[3:22])

輸出：

您可以根據需要處理此字符串數據。

Query Date: Wed Aug 09 2017
Latitude: 33.4484
Longitude: -112.0740

ASCE 7-10 Windspeeds
(3-sec peak gust in mph*):

Risk Category I: 105
Risk Category II: 115
Risk Category III-IV: 120
MRI** 10-Year: 76
MRI** 25-Year: 84
MRI** 50-Year: 90
MRI** 100-Year: 96

ASCE 7-05 Windspeed:
90 (3-sec peak gust in mph)
ASCE 7-93 Windspeed:
72 (fastest mile in mph)