我怎样才能为获奖者网站抓取一个网站

Question

Hi I am trying to scrape this website with Python 3 and noticed that in the source code it does not give a clear indication of how I would scrape the names of the winners in these primary elections. Can you show me how to scrape a list of all the winners in every MD primary election with this website?

https://elections2018.news.baltimoresun.com/results/

Answer 1

The parsing is a little bit complicated, because the results are in many subpages. This scripts collects them and prints result (all data is stored in variable data ):

from bs4 import BeautifulSoup
import requests

url = "https://elections2018.news.baltimoresun.com/results/"
r = requests.get(url)

data = {}
soup = BeautifulSoup(r.text, 'lxml')
for race in soup.select('div[id^=race]'):
    r = requests.get(f"https://elections2018.news.baltimoresun.com/results/contests/{race['id'].split('-')[1]}.html")
    s = BeautifulSoup(r.text, 'lxml')
    l = []
    data[(s.find('h3').text, s.find('div', {'class': 'party-header'}).text)] = l

    for candidate, votes, percent in zip(s.select('td.candidate'), s.select('td.votes'), s.select('td.percent')):
        l.append((candidate.text, votes.text, percent.text))

print('Winners:')
for (race, party), v in data.items():
    print(race, party, v[0])

# print(data)

Outputs:

Winners:
Governor / Lt. Governor Democrat ('Ben Jealous and Susan Turnbull', '227,764', '39.6%')
U.S. Senator Republican ('Tony Campbell', '50,915', '29.2%')
U.S. Senator Democrat ('Ben Cardin', '468,909', '80.4%')
State's Attorney Democrat ('Marilyn J. Mosby', '39,519', '49.4%')
County Executive Democrat ('John "Johnny O" Olszewski, Jr.', '27,270', '32.9%')
County Executive Republican ('Al Redmer, Jr.', '17,772', '55.7%')