Extract data with BeautifulSoup

Question

I need to extract "Ended 7 seconds ago" from file:

<div class="featured__columns">             
                            <div class="featured__column"><i style="color:rgb(149,213,230);" class="fa fa-clock-o"></i> <span title="Today, 11:49am">Ended 7 seconds ago</span></div>
                            <div class="featured__column featured__column--width-fill text-right"><span title="March 7, 2016, 10:50am">2 days ago</span> by <a style="color:rgb(149,213,230);" href="/user/Eclipsy">Eclipsy</a></div><a href="/user/Eclipsy" class="global__image-outer-wrap global__image-outer-wrap--avatar-small">
                                <div class="global__image-inner-wrap" style="background-image:url(https://steamcdn-a.akamaihd.net/steamcommunity/public/images/avatars/dc/dc5b8424bd5d17e13dcfe613689921dfc29f4574_medium.jpg);"></div>
                            </a>
                        </div>

I try:

#!/usr/bin/python3
from bs4 import BeautifulSoup
with open("./source.html") as source_html:
    soup=BeautifulSoup(source_html.read())
    soup=soup.find_all("span")
    print(soup[0].string)

All good, but i think my method so stupid. There are different way to extract data?

Answer 1

The span you want is in the first featured__column div :

from bs4 import BeautifulSoup

html ="""<div class="featured__columns">
                            <div class="featured__column"><i style="color:rgb(149,213,230);" class="fa fa-clock-o"></i> <span title="Today, 11:49am">Ended 7 seconds ago</span></div>
                            <div class="featured__column featured__column--width-fill text-right"><span title="March 7, 2016, 10:50am">2 days ago</span> by <a style="color:rgb(149,213,230);" href="/user/Eclipsy">Eclipsy</a></div><a href="/user/Eclipsy" class="global__image-outer-wrap global__image-outer-wrap--avatar-small">
                                <div class="global__image-inner-wrap" style="background-image:url(https://steamcdn-a.akamaihd.net/steamcommunity/public/images/avatars/dc/dc5b8424bd5d17e13dcfe613689921dfc29f4574_medium.jpg);"></div>
                            </a>
                        </div>"""


print(BeautifulSoup(html).select("div.featured__column span")[0].text)
Ended 7 seconds ago

If you want the first specifically, or the nth span you can use nth-of-type in the select:

In [53]: BeautifulSoup(html).select("div.featured__column span")
Out[53]: 
[<span title="Today, 11:49am">Ended 7 seconds ago</span>,
 <span title="March 7, 2016, 10:50am">2 days ago</span>]

In [54]: BeautifulSoup(html).select("div.featured__column span:nth-of-type(1)")
Out[54]: [<span title="Today, 11:49am">Ended 7 seconds ago</span>]

In [55]: BeautifulSoup(html).select("div.featured__column span:nth-of-type(2)")
Out[55]: [<span title="March 7, 2016, 10:50am">2 days ago</span>]
In [56]: BeautifulSoup(html).select("div.featured__column span:nth-of-type(2)")[0].text
Out[56]: u'2 days ago'

In [57]: BeautifulSoup(html).select("div.featured__column span:nth-of-type(1)")[0].text
Out[57]: u'Ended 7 seconds ago'

We can also use the i tag with the class fa fa-clock-o and get it's adjacent sibling span:

In [70]: BeautifulSoup(html).select("i.fa.fa-clock-o + span")
Out[70]: [<span title="Today, 11:49am">Ended 7 seconds ago</span>]

In [71]: BeautifulSoup(html).select("i.fa.fa-clock-o + span")[0].text
Out[71]: u'Ended 7 seconds ago'

lastly to exactly replicate your own logic and get just the first span html regardless of class etc.. you could simplify to:

BeautifulSoup(html).select("span:nth-of-type(1)")[0].text
BeautifulSoup(html).find("span").text

Answer 2

You can try something like

f_c = soup.find_all('div', class='featured__columns')[0]
print f_c.find('div', class='featured__column').span.get_text()

Similarly, if there are multiple div tags with class featured__columns then you can loop through it and fetch your data.