简体   繁体   中英

How do you grab specific "font-sizes" while ignoring others using BeautifulSoup in Python?

I am currently scraping a website and need to obtain certain font-sizes, such that in style="font-size: 140%", I want to grab 140% or preferably just 140 so that I can use it in certain calculations since each one will have a different font-size.

More specifically, I want to grab the font-sizes from tags like this...

 <div style="font-size: 141%; line-height: 110%"><a href="engenremap-latinrock.html" style="color: #AF7E1C">latin rock</a></div> <div style="font-size: 139%; line-height: 110%"><a href="engenremap-mexicanindie.html" style="color: #B18230">mexican indie</a></div>

Typically I'd be able to do this with no problem. However, the issue I am having is that there are no distinguishing tags on what I want to grab and there are a bunch of preceding lines consisting of tags with font-size like this...

 <tr valign=top class="datarow firstrow" style="white-space: nowrap"><td align=right class=note style="font-size: 20px; line-height: 24px">1</td><td style="font-size: 20px; line-height: 24px"> <a href="spotify:playlist:0DsV1U8e3xXsmsDSaW88XT" class=note target=spotify title="See this playlist in Spotify.">&#x260A;</a></td><td class=note style="font-size: 20px; line-height: 24px"><a href="?scope=MX&vector=activity" title="Show only schools from Mexico." style="color: #BA890D">Mexico</a></td><td class=note style="font-size: 20px; line-height: 24px"><a href="?root=Universidad%20Nacional%20Aut%C3%B3noma%20De%20M%C3%A9xico%20%28UNAM%29&scope=all" title="Re-sort the list by similarity to Universidad Nacional Autónoma De México (UNAM)." style="color: #BA890D">Universidad Nacional Autónoma De México (UNAM)</a></td></tr> <tr valign=top class="datarow " style="white-space: nowrap"><td align=right class=note style="font-size: 20px; line-height: 24px">2</td><td style="font-size: 20px; line-height: 24px"> <a href="spotify:playlist:5QAomgXhxwYjg975DWtTTv" class=note target=spotify title="See this playlist in Spotify.">&#x260A;</a></td><td class=note style="font-size: 20px; line-height: 24px"><a href="?scope=US&vector=activity" title="Show only schools from USA." style="color: #948F04">USA</a></td><td class=note style="font-size: 20px; line-height: 24px"><a href="?root=Texas%20A%20%26%20M%20University-College%20Station&scope=all" title="Re-sort the list by similarity to Texas A & M University-College Station." style="color: #948F04">Texas A &amp; M University-College Station</a></td></tr>

Keeping in mind that I already traverse the links in the previous snippet (they are static and remain in the same position as I traverse) and that the tags from the first snippet (genres at schools) change for every link (schools), how would I go about ignoring the font-sizes from the tr tags and only grab the font-sizes from the first HTML snippet? I'm sure this has a simple solution, but I'd appreciate any help.

** I am already traversing through the links and grabbing the respective genres for each school, I just need to also grab the font-sizes for those specific genres as well. **

Here is some of my code to provide more context...

data = []   # used to sort between country and university <td> tags
links = []  # stores links from clicking on the university name and used to get genres

countries = []          #
universities = []       # indices match for these lists
spotifyLinks = []       #
fontSizes = []          #
genres = [[]]           #
genres_weight = [[]]    #

page = requests.get("http://everynoise.com/everyschool.cgi")    # stores response from Every Noise into page
soup = BeautifulSoup(page.content, 'html.parser')   # used to create data list
soup1 = BeautifulSoup(page.content, 'html.parser')  # used to create links and spotifyLinks lists

soupList = list(soup.find_all('td', class_="note")) # creates list of <td> tags where class="note"

for soup in soupList:                   #
    if not soup.get_text().isnumeric(): # stores all country and university names in data list
        data.append(soup.get_text())    #

for i in range(len(data)):              #
    if i%2 == 0:                        # separates data list into two individual lists
        countries.append(data[i])       # for country and university names respectively
    else:                               #
        universities.append(data[i])    #

for a in soup1.find_all('a', attrs={'href': re.compile("\?root=")}):
    links.append('http://everynoise.com/everyschool.cgi' + a['href'])

for a in soup1.find_all('a', attrs={'href': re.compile("spotify:playlist:")}):
    spotifyLinks.append('https://open.spotify.com/playlist/' + a['href'][17:])

spotifyLinks = spotifyLinks[:-1]

linkSubset = links[0:4] # subset of links for quicker testing
j=1

for link in linkSubset: # switch out linkSubset with links for full dataset
    time.sleep(1)   # so we don't spam their servers
    schoolGenres = []
    nextPage = urllib.urlopen(url=link)
    bs_obj = BeautifulSoup(nextPage, "html.parser")

    for a in bs_obj.find_all('a', attrs={'href': re.compile("^engenremap-")}):
        schoolGenres.append(a.get_text())

    genres.append(schoolGenres)
    print "Scraping...", j
    j=j+1

genres = genres[1:]
distinct_genres = set()

for genre in genres:
    distinct_genres.update(genre)

print "\nDistinct Genres:", distinct_genres

Edit/Answer: This ended up being solved by using a slightly modified version of the answer selected.

pattern = re.compile(r'font-size: (\d+)')

for a in bs_obj.select('div[style*="font-size"]'):
    genreWeights.append(int(pattern.search(str(a)).group(1)))

You can search for a tag that contains certain text, and than extract the value of font-size . For example:

import re
from bs4 import BeautifulSoup

txt = """<div style="font-size: 141%; line-height: 110%"><a href="engenremap-latinrock.html" style="color: #AF7E1C">latin rock</a></div>
<div style="font-size: 139%; line-height: 110%"><a href="engenremap-mexicanindie.html" style="color: #B18230">mexican indie</a></div>
<div style="font-size: 139%; line-height: 110%"><a href="engenremap-mexicanindie.html" style="color: #B18230">hello world/a></div>"""

soup = BeautifulSoup(txt, "html.parser")

pattern = re.compile(r'font-size: (\d+)')
for tag in soup.select("div:contains(latin, mexican)"):
    font_size = pattern.search(str(tag)).group(1)
    print(font_size)

Output:

141
139

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM