Python Web Scraping td类跨度

Question

New to Python and Web Scraping... I have been looking to scrape the highlighted section of code so I can retrieve the numbers 1.16, 7.50 and 14.67, but am having no joy in using a td, class, table-matches__odds pageSoup.find_all... anyone know what I'm missing here?

I'm using beautifulsoup 4.

Answer 1

Awkward.

First I found the column of 'ratio' items (odds?), as reference points within the rows we want to plunder. Put them in the list called ratio .

Then I had a look at the next siblings for a typical element of ratio , namely the first.

You're interested only in the first row of the table, therefore I picked up ratio[0] and asked for its next siblings, which are all td elements.

I then extracted what you want from each of these, depending on its internal stucture. The only complicated one was the first. I used the descendants iterator to get its descendants, asked for the innermost one, and then got that one's attribute.

>>> import bs4
>>> import requests
>>> page = requests.get('http://www.betexplorer.com/soccer/scotland/premiership-2016-2017/results/').text
>>> soup = bs4.BeautifulSoup(page, 'lxml')
>>> ratio = soup.findAll('td', attrs={'class': 'h-text-center'})
>>> ratio[0].findNextSiblings()
[<td class="table-matches__odds colored"><span><span><span data-odd="1.16"></span></span></span></td>, <td class="table-matches__odds" data-odd="7.50"></td>, <td class="table-matches__odds" data-odd="14.67"></td>, <td class="h-text-right h-text-no-wrap">21.05.2017</td>]
>>> len(ratio)
15
>>> zeroth_ratio_sibs = ratio[0].findNextSiblings()
>>> first_item = list(zeroth_ratio_sibs[0].descendants)[2].attrs['data-odd']
>>> first_item
'1.16'
>>> second_item = zeroth_ratio_sibs[1].attrs['data-odd']
>>> second_item
'7.50'
>>> third_item = zeroth_ratio_sibs[2].attrs['data-odd']
>>> third_item 
'14.67'