[英]Web scraping using beautiful soup (sports data)
When i try loading this code i get two errors. 当我尝试加载此代码时,出现两个错误。 1:The first one is that i can't properly scrape the data for name_text.
1:第一个是我无法正确抓取name_text的数据。
2: I get an indentation error for team = name_text.div.text. 2:我收到team = name_text.div.text的缩进错误。 I know it is probably easy to solve but i have tried different indentations and nothing seems to be working.
我知道这可能很容易解决,但是我尝试了不同的缩进,但似乎没有任何效果。
in the website I am wanting to scrape the team name and the odds. 在网站上,我想抓取球队名称和赔率。
<div class="size14_f7opyze Endeavour_fhudrb0 medium_f1wf24vo participantText_fivg86r" data-automation-id="participant-one">Orlando Magic</div>
<div class="priceText_f71sibe"><span class="size14_f7opyze medium_f1wf24vo priceTextSize_frw9zm9" data-automation-id="price-text">5.85</span></div>
The html above has been copied from the site. 上面的html已从网站复制。
from bs4 import BeautifulSoup
from urllib.request import urlopen as uReq
my_url = 'https://www.sportsbet.com.au/betting/basketball-us'
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
soup = BeautifulSoup(page_html, "html.parser")
price_text = soup.findAll("div",{"class":"priceText_f71sibe"})
name_text = soup.findAll("div",{"class":"size14_f7opyze Endeavour_fhudrb0 medium_f1wf24vo participantText_fivg86r"})
filename = "odds.csv"
f = open(filename,"w")
headers = "Team, odds_team\n"
print(name_text)
f.write(headers)
for price_text in price_texts:
team = name_text.div.text
odds = price_text.span.text
print(odds)
print(team + odds)
f.write(team + "," + odds + "\n")
f.close()
Any help would be great. 任何帮助都会很棒。 Cheers.
干杯。
Your for loop
indentation is not correct. 您的
for loop
缩进不正确。 The correct indentation would be: 正确的缩进将是:
for price_text in price_texts:
team = name_text.div.text
odds = price_text.span.text
team = name_text.div.text
odds = price_text.span.text
print(odds)
print(team + odds)
f.write(team + "," + odds + "\n")
f.close()
With 4 spaces before team and odds. 团队和赔率前有4个空格。 Please read the Python ForLoop documentation .
请阅读Python ForLoop文档 。
Also, there is no price_texts
variable. 而且,没有
price_texts
变量。 You need to assign it when you do findAll , you forgot an 'S': 当您执行findAll时 ,您需要分配它,您忘记了'S':
price_texts = soup.findAll("div",{"class":"priceText_f71sibe"})
Last thing, consider using with
instead of open()
and .close()
to write to your file. 最后一件事,请考虑使用
with
代替open()
和.close()
来写入文件。
I was thinking what you could do is just iterate and store those into lists, then write to file. 我在想,您可以做的只是迭代并将这些存储到列表中,然后写入文件。 I unfortunately can't access the site at work, so I can't test the code, but I believe this should give the output you are looking for:
不幸的是,我无法在工作中访问该网站,因此无法测试代码,但是我相信这应该可以提供您所需要的输出:
from bs4 import BeautifulSoup
from urllib.request import urlopen as uReq
import csv
from itertools import zip_longest
my_url = 'https://www.sportsbet.com.au/betting/basketball-us'
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
soup = BeautifulSoup(page_html, "html.parser")
price_text = soup.findAll("span",{"data-automation-id":"price-text"})
name_text = soup.findAll("div",{"data-automation-id":"participant-one"})
team_list = [ name.text.strip() for name in name_text ]
odds_list = [ price.text.strip() for price in price_text ]
d = [team_list, odds_list]
export_data = zip_longest(*d, fillvalue = '')
with open('odds.csv', 'w', encoding="ISO-8859-1", newline='') as myfile:
wr = csv.writer(myfile)
wr.writerow(("Team", "odds_team"))
wr.writerows(export_data)
myfile.close()
could you try this? 你可以试试这个吗?
from bs4 import BeautifulSoup
from urllib.request import urlopen as uReq
my_url = 'https://www.sportsbet.com.au/betting/basketball-us'
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
soup = BeautifulSoup(page_html, "html.parser")
price_texts = soup.findAll("div",{"class":"priceText_f71sibe"})
name_texts = soup.findAll("div",{"class":"size14_f7opyze Endeavour_fhudrb0 medium_f1wf24voparticipantText_fivg86r"})
filename = "odds.csv"
f = open(filename,"w")
headers = "Team, odds_team\n"
print(name_text)
f.write(headers)
odds =''
team=''
for price_text in price_texts:
odds = price_text.text
for name_text in name_texts:
team = name_text.text
print(odds)
print(team + odds)
f.write(team + "," + odds + "\n")
f.close()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.