![](/img/trans.png)
[英]I have python3.4 but no pip or ensurepip.. is something wrong with my python3.4 version?
[英]What is wrong with my web scraper code (python3.4)
我正在嘗試從網站上抓一張桌子。 它運行,但是我沒有輸出到我的文件。 我要去哪里錯了?
碼:
from bs4 import BeautifulSoup
import urllib.request
f = open('nbapro.txt','w')
errorFile = open('nbaerror.txt','w')
page = urllib.request.urlopen('http://www.numberfire.com/nba/fantasy/full-fantasy-basketball-projections')
content = page.read()
soup = BeautifulSoup(content)
tableStats = soup.find('table', {'class': 'data-table xsmall'})
for row in tableStats.findAll('tr')[2:]:
col = row.findAll('td')
try:
name = col[0].a.string.strip()
f.write(name+'\n')
except Exception as e:
errorFile.write (str(e) + '******'+ str(col) + '\n')
pass
f.close
errorFile.close
問題在於,您要抓取的表數據是通過在瀏覽器端調用javascript代碼來填寫的。 urllib
不是瀏覽器,因此無法執行javascript。
如果要通過urllib
和BeautifulSoup
解決它,則必須從script
標記中提取JSON對象,然后通過json.loads()
加載它。 示例,顯示玩家名稱:
import json
import re
import urllib.request
from bs4 import BeautifulSoup
soup = BeautifulSoup(urllib.request.urlopen('http://www.numberfire.com/nba/fantasy/full-fantasy-basketball-projections'))
script = soup.find('script', text=lambda x: x and 'NF_DATA' in x).text
data = re.search(r'NF_DATA = (.*?);', script).group(1)
data = json.loads(data)
for player_id, player in data['players'].items():
print(player['name'] + ' ' + player['last_name'])
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.