Web Scraping 一个动态网站，它使用 javasript 和漂亮的汤和正则表达式

Question

I am trying to make an app that gives fantasy football scores for the XFL as a personal project. I was able to use beautiful soup to get the source and String.split() to separate all the stats of the players in But when I try to get the rosters I get something like this:

>**1**</fagtd><td style="background-color:white; border-bottom:1px solid black; border-left:none; border-right:1px solid black; border-top:none; text-align:center; vertical-align:bottom; white-space:nowrap; width:89px">**Jazz**</td><td style="background-color:white; border-bottom:1px solid black; border-left:none; border-right:1px solid black; border-top:none; text-align:center; vertical-align:bottom; white-space:nowrap; width:100px">**Ferguson**</td><td style="background-color:white; border-bottom:1px solid black; border-left:none; border-right:1px solid black; border-top:none; text-align:center; vertical-align:bottom; white-space:nowrap; width:61px">**WR**

and out of this I need to get the information 1 Jazz Ferguson and WR . String.split() will not work for something this complex. I was thinking about using regular expressions but I am not sure how. Can any one come up with a reg ex for this or if there is a much easier way point me in the right direction? Thank you.

EDIT This is the portion of the code I use to get that HTML data above. It prints out the whole thing that part above is only a section.

session = HTMLSession()
page = session.get('https://www.xfl.com/en-US/teams/dallas/renegades-articles/dallas-renegades-roster')

soup2 = BeautifulSoup(page.content, PARSER)
script = soup2.find_all('script')

for tags in script:

    if ((tags.text.find('"title":"Dallas Renegades roster"')) >= 0):

        rosterData = tags.text[(tags.text.find('College')):]
        rosterData = rosterData.replace('</td>', '').replace('\\','')

        print(rosterData)

Answer 1

Hi below code gets the full table as a dataframe you can filter the required data from this:-

import requests
import pandas as pd
url = 'https://www.xfl.com/en-US/teams/dallas/renegades-articles/dallas-renegades-roster'
html = requests.get(url).content
df_list = pd.read_html(html)
df = df_list[-1]
print(df)