I am currently using python to web scrape the three-point statistics for every NBA player and am trying to put this data in a data frame. The code below is my attempt at adding the values to the data frame. The variables players,teams,threePointAttempts, and threePointPercentage are all lists containing 50 values. These are refilled after every iteration of the while loop because the script moves through each page of the NBA site.
while i<10:
soup = BeautifulSoup(d.page_source, 'html.parser').find('table')
headers, [_, *data] = [i.text for i in soup.find_all('th')], [[i.text for i in b.find_all('td')] for b in soup.find_all('tr')]
final_data = [i for i in data if len(i) > 1]
data_attrs = [dict(zip(headers, i)) for i in final_data]
print(data_attrs)
players = [i['PLAYER'] for i in data_attrs]
teams = [i['TEAM'] for i in data_attrs]
threePointAttempts = [i['3PA'] for i in data_attrs]
threePointPercentage = [i['3P%'] for i in data_attrs]
data_df = data_df.append(pd.DataFrame(players, columns=['Player']),ignore_index=True)
data_df = data_df.append(pd.DataFrame(teams, columns=['Team']),ignore_index=True)
data_df = data_df.append(pd.DataFrame(threePointAttempts, columns=['3PA']),ignore_index=True)
data_df = data_df.append(pd.DataFrame(threePointPercentage, columns=['3P%']),ignore_index=True)
data_df = data_df[['Player','Team','3PA','3P%']]
The issue I am having is the data frame fills like this:
Try:
temp_df = pd.DataFrame({'Player': players,
'Team': teams,
'3PA': threePointAttempts,
'3P%': threePointPercentage})
data_df = data_df.append(temp_df, ignore_index=True)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.