Python使用漂亮的汤从HTML标记中提取数字

Question

I am working on a web scraper using beautiful soup. Here is my function:

    journalist_result = soup.find_all("h4",class_="slab")
    if len(journalist_result)>0:
        journalist_share = int(re.match(r'\d+', journalist_result[0].get_text()).group())
    else:
        journalist_share=0

Basically, what I want to do is to extract the number of journalist that shared a link. In this case is 221 (see below for example):

CASE1:

<h4 class="slab">221 journalists shared this link.
      <a href="/pros">Join</a> or <a href="/account/login?next=/whosharedmylink/?url=http://www.cnn.com/">sign in</a> to Muck Rack to view their names.</h3>

My code works fine for cases where there are journalist shares or if an URL is not found. However, my code breaks on the following case:

CASE2:

<h4 class="slab" style="margin-bottom:5px">

      This link hasn't yet been shared by any journalists.<br /><a href="/pros">Learn about using Muck Rack Pro</a> to connect with journalists.
</h4>

this is because in case 2, there are no journalist found. And the error I am getting is:

Traceback (most recent call last): File "muckrackscraper.py", line 65, in journalist_share = int(re.match(r'\\d+', journalist_result[0].get_text()).group()) AttributeError: 'NoneType' object has no attribute 'group'

THanks in advance for any help!

Answer 1

It seems like you've misunderstood why your code is failing. It's not in case 2, but in case 1 where you don't check the return value from re.match and then attempt a function call on None .

From the re.match documentation :

Return None if the string does not match the pattern; note that this is different from a zero-length match.

So your pattern is not matching whatever is in journalist_result[0].get_text() ; try inspecting this value and also add a check for None .