使用beautifulsoup从span类标签中提取文本

Question

I am trying to extract some elements of text between a span class from a website.

Here is a snippet of the HTML code:

<h1>2 Some address</h1>
                </div>
                <div id="smi-summary-items">
                    <div id="smi-price-string">&euro;230,000</div>
                    <span class="header_text"> Detached House</span><span class="bar">&nbsp;|&nbsp;</span><span class="header_text">3 Beds</span><span class="bar">&nbsp;|&nbsp;</span><span class="header_text">2 Baths</span>
                    <!-- Text_Link_Full_Ad_Unit -->
                    <div id='dfp-text_link_full_ad_unit' class='sale'>
                        <script type='text/javascript'>
                            googletag.cmd.push(function()
                                {
                                    googletag.display('dfp-text_link_full_ad_unit');
                                }
                            );
                        </script>
                    </div>

I would like to extract the text of "3 Beds" and "2 Baths".

I've tried a few solutions but mainly getting errors or an empty result.

Can anyone suggest a solution?

Answer 1

From what I understand, you can simply filter the desired elements by class:

[item.get_text() for item in soup.select("span.header_text")]

---

The complete working sample code:

from bs4 import BeautifulSoup

data = """
<div id="smi-summary-items">
    <div id="smi-price-string">&euro;230,000</div>
    <span class="header_text"> Detached House</span><span class="bar">&nbsp;|&nbsp;</span><span class="header_text">3 Beds</span><span class="bar">&nbsp;|&nbsp;</span><span class="header_text">2 Baths</span>
    <!-- Text_Link_Full_Ad_Unit -->
    <div id='dfp-text_link_full_ad_unit' class='sale'>
        <script type='text/javascript'>
            googletag.cmd.push(function()
                {
                    googletag.display('dfp-text_link_full_ad_unit');
                }
            );
        </script>
    </div>"""
soup = BeautifulSoup(data, "html.parser")
print([item.get_text(strip=True) for item in soup.select("span.header_text")])

That produces:

['Detached House', '3 Beds', '2 Baths']

Answer 2

The following code works for extracting some elements of text between a span class from a website

>>> from bs4 import BeautifulSoup
>>> import re
>>> content = """<h1>2 Some address</h1>
...                 </div>
...                 <div id="smi-summary-items">
...                     <div id="smi-price-string">&euro;230,000</div>
...                     <span class="header_text"> Detached House</span>
<span class="bar">&nbsp;|&nbsp;</span><span class="header_text">3 
Beds</span><span class="bar">&nbsp;|&nbsp;</span><span class="header_text">2 
Baths</span>
...                     <!-- Text_Link_Full_Ad_Unit -->
...                     <div id='dfp-text_link_full_ad_unit' class='sale'>
...                         <script type='text/javascript'>
...                             googletag.cmd.push(function()
...                                 {
...                                     googletag.display('dfp-
text_link_full_ad_unit');
...                                 }
...                             );
...                         </script>
...                     </div>"""

>>> soup = BeautifulSoup(content, "html.parser")
>>> req = soup.find_all("span", {"class":"header_text"})
>>> print(req)
[<span class="header_text"> Detached House</span>, <span 
class="header_text">3 Beds</span>, <span class="header_text">2 Baths</span>]
>>> x23 = []
>>> for i in req:
...     x23.append(i.get_text())
...
>>> print(x23)
[' Detached House', '3 Beds', '2 Baths']