How to store items separated by <br> into separate arrays using Python BeautifulSoup?

Question

Trying to scrape climbing gym data. I'm using BeautifulSoup. I want to store arrays of the gym name, location, phone number, link, and description.

Here is sample html:

div class="city">Alberta</div>
<p><b>Camp He Ho Ha Climbing Gym</b><br>
Seba Beach, Alberta, TOE 2BO Canada<br>
(780) 429-3277<br>
<a rel='nofollow' target='_blank' href='http://camphehoha.com/summer-camp/camp-life/'>Camp He Ho Ha Climbing Gym</a><br>
<span class='rt'></span> The Summit is Camp He Ho Ha's 40' climbing gym and ropes course. Facility is available for rent, with safety equipment, orientation to the course and staffing provided.</p>
<div class="city">Calgary</div>
<p><b>Bolder Climbing Community</b><br>
5508 1st Street SE, Calgary, Alberta, Canada<br>
403 988-8140<br>
<a rel='nofollow' target='_blank' href='http://www.bolderclimbing.com/'>Bolder Climbing Community</a><br>
<span class='rt'></span> Calgary's first bouldering specific climbing centre.</p>

I can easily move between each climbing gym because they are separated by <p> but the individual items I'm interested in are separated by <br> . How do I store these items into separate arrays?

Answer 1

You can do something like this. Basically, find the <br> tag, then the content right before it.

html = '''div class="city">Alberta</div>
<p><b>Camp He Ho Ha Climbing Gym</b><br>
Seba Beach, Alberta, TOE 2BO Canada<br>
(780) 429-3277<br>
<a rel='nofollow' target='_blank' href='http://camphehoha.com/summer-camp/camp-life/'>Camp He Ho Ha Climbing Gym</a><br>
<span class='rt'></span> The Summit is Camp He Ho Ha's 40' climbing gym and ropes course. Facility is available for rent, with safety equipment, orientation to the course and staffing provided.</p>
<div class="city">Calgary</div>
<p><b>Bolder Climbing Community</b><br>
5508 1st Street SE, Calgary, Alberta, Canada<br>
403 988-8140<br>
<a rel='nofollow' target='_blank' href='http://www.bolderclimbing.com/'>Bolder Climbing Community</a><br>
<span class='rt'></span> Calgary's first bouldering specific climbing centre.</p>'''

from bs4 import BeautifulSoup


soup = BeautifulSoup(html, 'html.parser')
final_content = []
ps = soup.find_all('p')
for p in ps:
    content = []
    breaks = p.find_all('br')
    for br in breaks:
        try:
            b = br.previousSibling.strip()
            content.append(b)
        except:
            continue
    final_content.append(content)

Output:

print (final_content)
[['Seba Beach, Alberta, TOE 2BO Canada', '(780) 429-3277'], ['5508 1st Street SE, Calgary, Alberta, Canada', '403 988-8140']]