Extract data between br Tag with beautifulsoup

Question

how can I extract INFO1 and INFO2 from following html code with beautifulsoup?

webiste: https://swisswrestling.ch/wrestlers?id=91

I try also to extract data for the fights but that works for me with bs4.

    <tr>
<td>
<span class="text-danger font-weight-bold"><br/>
<b>Notice</b>:  Undefined index: HTTP_ACCEPT_LANGUAGE in <b>/srv/www/chroot/site05/web/app/bootstrap.php</b> on line <b>159</b><br/>
wrestlers_club</span> INFO1<br/>
<span class="text-danger font-weight-bold">wrestlers_birthday</span> INFO2<br/><br/>
<span class="text-danger font-weight-bold">wrestlers_licence_number</span> wrestlers_licence_no_licence<br/>
<span class="text-danger font-weight-bold">wrestlers_club_dl</span> wrestlers_licence_no_dl                                                
                    </td>

My code looks now:

info = soup.find('div', id='content')
club = info.findAll('span')
for clubs in club:
 test = clubs.text
 print(test)

And the Result is:

Notice :

 Undefined index: HTTP_ACCEPT_LANGUAGE in /srv/www/chroot/site05/web/app/bootstrap.php on line 159

wrestlers_club

wrestlers_birthday

wrestlers_licence_number

wrestlers_club_dl

How can I extract the data behind wreslters_club (INFO1) and wrestlers_birthday (INFO2) ?

Thanks for your help!

Answer 1

Use following Css selector and find_next_sibling(text=True)

    import requests
    from bs4 import BeautifulSoup

    res=requests.get("https://swisswrestling.ch/wrestlers?id=91")
    soup=BeautifulSoup(res.text,'lxml')
    print(soup.select_one('span.text-danger:nth-of-type(1)').find_next_sibling(text=True).strip())
    print(soup.select_one('span.text-danger:nth-of-type(2)').find_next_sibling(text=True).strip())

Output :

RC Willisau Lions
29 (1990)

Answer 2

Does this work for you? I used the link you posted and it extracts the required information.

import requests
import re
import lxml
import ssl
from bs4 import BeautifulSoup as bs
ssl._create_default_https_context = ssl._create_unverified_context
url = 'https://swisswrestling.ch/wrestlers?id=91'

strx = requests.get(url).text
regex = r"(wrestlers_club|wrestlers_birthday) (.*)\n"
soup = bs(strx, 'lxml')


for i in soup.find_all('td'):
    print (*re.findall(regex,i.text), sep="\n")

Output:

('wrestlers_club', 'RC Willisau Lions')
('wrestlers_birthday', '29 (1990)')

Answer 3

You can try this, it gave me the result you requested:

     info = soup.find('div', id='content')
     club = info.select('#content-element-94 > div.row.border.mx-0 > div > table > tbody > tr > td:nth-child(1)')
     for clubs in club:
       test = clubs.text
       print(test)

output:

       wrestlers_club RC Willisau Lions
       wrestlers_birthday 29 (1990)
       wrestlers_licence_number wrestlers_licence_no_licence
       wrestlers_club_dl wrestlers_licence_no_dl