网页抓取数据表以取得优异成绩

Question

I'm trying to scrape data from a site to excel. right now it's working fine but when it searches someone like Sergio Rodriguez, multiple names come up ( https://basketball.realgm.com/search?q=Sergio+Rodriguez ), so it skips the name and throws out "No international table for Sergio Rodriguez." How do I select the one that played in the NBA from that list and continue on with scraping the per game and advanced stats tables to excel? in this case, rodriguez is second when you search his name.

import requests
from bs4 import BeautifulSoup
import pandas as pd


playernames=['Carlos Delfino', 'Sergio Rodriguez']

result = pd.DataFrame()
for name in playernames:

    fname=name.split(" ")[0]
    lname=name.split(" ")[1]
    url="https://basketball.realgm.com/search?q={}+{}".format(fname,lname)
    response = requests.get(url)

    soup = BeautifulSoup(response.content, 'html.parser')


    try:
        table1 = soup.find('h2',text='International Regular Season Stats - Per Game').findNext('table')
        table2 = soup.find('h2',text='International Regular Season Stats - Advanced Stats').findNext('table')

        df1 = pd.read_html(str(table1))[0]
        df2 = pd.read_html(str(table2))[0]

        commonCols = list(set(df1.columns) & set(df2.columns))
        df = df1.merge(df2, how='left', on=commonCols)
        df['Player'] = name

    except:
        print ('No international table for %s.' %name)
        df = pd.DataFrame([name], columns=['Player'])

    result = result.append(df, sort=False).reset_index(drop=True)

cols = list(result.columns)
cols = [cols[-1]] + cols[:-1]
result = result[cols]
result.to_csv('international players.csv', index=False)

Answer 1

Check the URL of the page you receive, a search which results in a single match directs you to

https://basketball.realgm.com/player/{player-name}/Summary/{player-id}

but when there is more than one result you get

https://basketball.realgm.com/search?q={player-name}

Write a parser function for both urls, such as (pseudocode)

...
for name in playernames:
    fname=name.split(" ")[0]
    lname=name.split(" ")[1]
    url="https://basketball.realgm.com/search?q={}+{}".format(fname,lname)
    response = requests.get(url)
    soup = BeautifulSoup(response.content, 'html.parser')
    # check the response url
    if (response.url == "https://basketball.realgm.com/search..."):
        # parse the search results, finding the players you want
        ... get urls from the table ...
        soup.table...  # etc.
        foreach url in table:
            response = requests.get(player_url)
            soup = BeautifulSoup(response.content, 'html.parser')
            # call the parse function for a player page
            ...
            parse_player(soup)
    else: # we have a player page
        # call the parse function for a player page, same as above
        ...
        parse_player(soup)
    ...

There's a small amount of code duplication, but while you get your head around it and make it work don't worry about that.