[英]Trying to webscrape table data from rotogrinders. Any help would be greatly appreciated
到目前为止，我有：
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.by import By
from bs4 import BeautifulSoup
import pandas as pd
driver = webdriver.Firefox()
url = r"https://rotogrinders.com/game-stats/nba-player?site=fanduel&range=yesterday"
driver.get(url)
cookies = driver.find_element_by_xpath(r'//*[@id="bc-close-cookie"]').click()
select = driver.find_element_by_xpath(r'/html/body/div[1]/div/section/div/section/div[2]/div[2]/a[2]').click()
我需要将表数据刮到 a.csv 文件中。 有什么建议么？
因为订阅弹窗。 订阅弹窗
发生此错误时，请在单击全部按钮之前单击不谢谢。
Selenium 在这里太过分了，因为数据在<script>标签内。 您可以使用requests通过一个简单的请求来获取它，然后 a) 使用 BeautifulSoup 拉出<script>标记，然后解析它，或者 b) 使用正则表达式直接拉出它。 我选择了后者。
它采用漂亮的 json 格式，这意味着使用pandas转换成表格相对容易。 您还将使用pnadas写入 csv。
import requests
import pandas as pd
import re
import json
url = 'https://rotogrinders.com/game-stats/nba-player?site=fanduel&range=yesterday'
response = requests.get(url)
p = re.compile("var data = (\[.*\])")
result = p.search(response.text)
jsonStr = result.group(1)
jsonData = json.loads(jsonStr)
df = pd.DataFrame(jsonData)
df.to_csv('nba.csv', index=False)
Output：
print(df.to_string())
id name fpts gp fgm fga ftm fta 3pm 3pa 2pm 2pa reb ast stl blk to pts oreb pfoul tfoul ffoul min fouls dd td usg pace fg% ft% 3p% 2p% team pos player
0 947 Lance Stephenson 6.70 1 1 1 2 2 0 0 1 1 1 3 0 0 3 4 0 1 0 0 17.80 0 0 0 13.19 5 1 1 0 1 IND SCW Lance Stephenson
1 1079 Stephen Curry 58.00 1 12 27 9 9 6 16 6 11 5 8 1 0 2 39 0 3 0 0 43.95 0 0 0 32.40 33.50 0.44 1 0.38 0.55 GSW CG Stephen Curry
2 1087 Chris Paul 51.50 1 8 14 2 2 2 6 6 8 5 11 2 1 0 20 0 3 0 0 36.68 0 1 0 20.19 15 0.57 1 0.33 0.75 PHO DIS Chris Paul
3 1277 Taj Gibson 12.50 1 1 3 0 0 0 0 1 3 5 1 0 1 0 2 1 4 0 0 17.95 0 0 0 7.42 2 0.33 0 0 0.33 NYK PB Taj Gibson
4 1334 Andre Iguodala 24.00 1 1 2 2 2 0 0 1 2 5 4 0 4 4 4 2 2 0 0 31.32 0 0 0 10.47 5 0.50 1 0 0.50 GSW 3DW Andre Iguodala
5 1485 JaVale McGee 10.80 1 3 5 2 2 0 0 3 5 4 0 0 0 2 8 1 3 0 0 17.82 0 0 0 17.69 7 0.60 1 0 0.60 PHO PB JaVale McGee
6 13301 Jonas Valanciunas 39.00 1 8 11 1 2 1 4 7 7 10 2 1 2 3 18 1 5 0 0 33.03 0 1 0 18.82 14 0.73 0.50 0.25 1 NOP PB Jonas Valanciunas
7 13312 Bismack Biyombo 19.30 1 2 3 5 7 0 0 2 3 4 1 2 0 2 9 2 3 0 0 27.90 0 0 0 12.06 6.50 0.67 0.71 0 0.67 PHO PB Bismack Biyombo
8 13315 Klay Thompson 13.90 1 6 17 0 0 0 7 6 10 2 1 0 0 2 12 1 1 0 0 23.10 0 0 0 33.47 18 0.35 0 0 0.60 GSW 3DW Klay Thompson
9 13335 Kemba Walker 11.40 1 1 5 3 4 0 3 1 2 2 2 1 0 2 5 0 2 1 0 21.17 0 0 0 17.80 9 0.20 0.75 0 0.50 NYK CG Kemba Walker
10 13353 Alec Burks 19.60 1 3 9 5 5 2 6 1 3 3 2 1 0 3 13 1 3 0 0 22.58 0 0 0 26.32 13.50 0.33 1 0.33 0.33 NYK CG Alec Burks
11 13913 Evan Fournier 10.10 1 2 8 1 2 1 6 1 2 3 1 0 0 1 6 1 1 0 0 25.15 0 0 0 16.24 9 0.25 0.50 0.17 0.50 NYK SHW Evan Fournier
12 13942 Jae Crowder 21.60 1 4 9 2 2 3 6 1 3 3 0 2 0 1 13 0 3 0 0 32.65 0 0 0 13.33 11 0.44 1 0.50 0.33 PHO 3DW Jae Crowder
13 13955 Jeremy Lamb 27.40 1 2 5 8 10 2 3 0 2 2 2 2 1 1 14 1 2 0 0 18.88 0 0 0 23.43 10 0.40 0.80 0.67 0 IND SCW Jeremy Lamb
14 14077 Garrett Temple 0.20 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 10.88 0 0 0 3.68 1 0 0 0 0 NOP SHW Garrett Temple
15 14559 Justin Holiday 23.50 1 6 13 0 0 4 9 2 4 5 3 0 0 3 16 1 2 0 0 34.22 0 0 0 19.86 15 0.46 0 0.44 0.50 IND SHW Justin Holiday
16 16879 Tim Hardaway Jr. 18.50 1 4 10 0 0 3 7 1 3 5 1 0 0 0 11 0 3 0 0 28.28 0 0 0 14.61 10 0.40 0 0.43 0.33 DAL SHW Tim Hardaway Jr.
17 16943 Reggie Bullock 11.70 1 1 3 0 0 1 3 0 0 6 1 0 0 0 3 1 1 0 0 20.18 0 0 0 6.60 2 0.33 0 0.33 0 DAL 3DW Reggie Bullock
18 18566 Dwight Powell 8.60 1 2 3 2 3 0 0 2 3 3 0 0 0 1 6 2 1 0 0 14.35 0 0 0 14.83 3.50 0.67 0.67 0 0.67 DAL VB Dwight Powell
19 18620 Andrew Wiggins 22.80 1 5 15 0 1 1 6 4 9 4 2 1 1 2 11 2 5 0 0 38.03 0 0 0 19.04 15.50 0.33 0 0.17 0.44 GSW SCW Andrew Wiggins
20 18632 Julius Randle 24.40 1 1 9 2 4 0 2 1 7 7 6 1 1 3 4 2 1 1 0 29.48 0 0 0 21.36 12 0.11 0.50 0 0.14 NYK VB Julius Randle
21 18899 Kristaps Porzingis 37.70 1 7 15 2 2 2 4 5 11 11 1 0 2 1 18 2 5 0 0 32.28 0 1 0 21.33 15 0.47 1 0.50 0.45 DAL VB Kristaps Porzingis
22 18941 Cameron Payne 18.60 1 4 10 3 3 1 4 3 6 3 0 1 0 0 12 0 1 1 0 18.93 0 0 0 23.92 11.50 0.40 1 0.25 0.50 PHO DIS Cameron Payne
23 18945 Kevon Looney 36.50 1 5 6 3 4 0 0 5 6 15 3 2 0 5 13 6 3 0 1 28.17 0 1 0 19.52 7 0.83 0.75 0 0.83 GSW PB Kevon Looney
24 18949 Devin Booker 41.00 1 11 25 5 5 1 8 10 17 5 6 0 0 2 28 0 2 1 0 38.30 0 0 0 32.56 29.50 0.44 1 0.13 0.59 PHO SCW Devin Booker
25 31814 Nemanja Bjelica 22.20 1 4 5 0 0 0 1 4 4 6 2 1 1 2 8 1 1 0 0 14.13 0 0 0 21.68 6 0.80 0 0 1 GSW VF Nemanja Bjelica
26 35227 Brandon Ingram 30.00 1 4 10 6 8 1 1 3 9 5 6 1 0 3 15 0 2 0 0 26.88 0 0 0 27.53 17 0.40 0.75 1 0.33 NOP SCW Brandon Ingram
27 35995 Damion Lee 14.00 1 3 5 2 2 1 2 2 3 0 0 2 0 1 9 0 3 0 0 14 0 0 0 19.66 7 0.60 1 0.50 0.67 GSW CG Damion Lee
28 36032 Dorian Finney-Smith 14.20 1 2 6 0 0 1 5 1 1 6 2 0 0 1 5 1 4 0 0 32 0 0 0 9.57 6 0.33 0 0.20 1 DAL 3DW Dorian Finney-Smith
29 36041 Gary Payton II 18.00 1 3 5 0 1 0 2 3 3 5 0 2 0 0 6 1 1 1 0 17.40 0 0 0 12.51 4.50 0.60 0 0 1 GSW CG Gary Payton II
30 37796 Frank Ntilikina 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1.70 0 0 0 0.00 0 0 0 0 0 DAL DIS Frank Ntilikina
31 37845 Maxi Kleber 22.00 1 3 9 2 2 1 7 2 2 5 4 0 1 2 9 1 2 0 0 27.13 0 0 0 19.46 11 0.33 1 0.14 1 DAL VB Maxi Kleber
32 37861 Torrey Craig 32.90 1 5 10 0 0 2 5 3 5 7 1 3 1 1 12 0 3 0 0 34.22 0 0 0 13.24 11 0.50 0 0.40 0.60 IND 3DW Torrey Craig
33 37889 Josh Hart 34.40 1 5 10 5 9 2 4 3 6 7 4 1 0 0 17 1 4 0 0 35.28 0 0 0 17.32 13.50 0.50 0.56 0.50 0.50 NOP SHW Josh Hart
34 408821 Mitchell Robinson 37.00 1 6 7 5 10 0 0 6 7 15 0 0 1 1 17 7 3 0 0 30.05 0 1 0 16.51 6 0.86 0.50 0 0.86 NYK PB Mitchell Robinson
35 408841 Landry Shamet 4.20 1 0 2 0 0 0 2 0 0 1 0 1 0 0 0 0 3 0 0 10.07 0 0 0 7.94 2 0 0 0 0 PHO SHW Landry Shamet
36 408842 Mikal Bridges 42.60 1 5 11 2 2 0 4 5 7 8 6 4 0 0 12 2 1 0 0 37.18 0 0 0 14.91 10 0.45 1 0 0.71 PHO 3DW Mikal Bridges
37 408968 Devonte' Graham 19.10 1 5 16 1 1 4 10 1 6 3 1 0 0 1 15 0 1 0 0 28.03 0 0 0 25.36 17.50 0.31 1 0.40 0.17 NOP CG Devonte' Graham
38 408971 Luka Doncic 44.60 1 9 23 8 11 2 9 7 14 8 8 1 0 8 28 1 1 0 0 38.13 0 0 0 40.37 35.50 0.39 0.73 0.22 0.50 DAL CG Luka Doncic
39 409062 Jalen Brunson 20.50 1 8 13 2 2 1 2 7 11 5 1 0 0 6 19 2 3 0 0 34.75 0 0 0 23.26 18 0.62 1 0.50 0.64 DAL DIS Jalen Brunson
40 570141 Gary Clark 3.50 1 1 5 0 0 1 4 0 1 0 1 0 0 1 3 0 0 0 0 7.95 0 0 0 31.85 6 0.20 0 0.25 0 NOP VF Gary Clark
41 1115150 RJ Barrett 30.20 1 6 13 4 7 1 5 5 8 6 2 0 1 0 17 1 1 0 0 27.98 0 0 0 23.93 15.50 0.46 0.57 0.20 0.63 NYK SCW RJ Barrett
42 1115261 Goga Bitadze 32.30 1 5 12 2 5 1 4 4 8 9 5 0 1 2 13 4 2 2 0 31.35 0 0 0 22.78 12.50 0.42 0.40 0.25 0.50 IND VB Goga Bitadze
43 1115443 Jaxson Hayes 8.90 1 3 4 0 0 0 0 3 4 2 1 0 0 1 6 0 5 0 0 16.47 0 0 0 12.94 5 0.75 0 0 0.75 NOP PB Jaxson Hayes
44 1115445 Nickeil Alexander-Walker 13.60 1 1 4 2 2 0 0 1 4 3 4 0 0 0 4 0 3 0 0 24.42 0 0 0 10.16 5 0.25 1 0 0.25 NOP CG Nickeil Alexander-Walker
45 1115568 Jordan Poole 13.40 1 1 7 3 4 0 5 1 2 2 4 0 0 0 5 0 2 0 0 24.67 0 0 0 16.34 9 0.14 0.75 0 0.50 GSW CG Jordan Poole
46 1115625 Cam Johnson 19.90 1 2 7 2 2 1 6 1 1 7 1 1 0 0 7 0 2 0 0 20.47 0 0 0 16.04 8 0.29 1 0.17 1 PHO SHW Cam Johnson
47 1311751 Oshae Brissett 12.70 1 0 7 2 4 0 3 0 4 6 1 1 0 1 2 0 1 0 0 21.98 0 0 0 18.36 10 0 0.50 0 0 IND SHW Oshae Brissett
48 1333886 Juan Toscano-Anderson 15.30 1 2 5 0 0 1 3 1 2 4 1 2 0 2 5 1 4 0 0 14.87 0 0 0 19.72 6 0.40 0 0.33 0.50 GSW SHW Juan Toscano-Anderson
49 2439136 Obi Toppin 5.40 1 0 1 0 0 0 1 0 0 2 0 0 1 0 0 0 1 0 0 18.52 0 0 0 2.16 1 0 0 0 0 NYK VB Obi Toppin
50 2439366 Josh Green 4.70 1 1 3 0 0 0 0 1 3 1 1 0 0 0 2 0 1 0 0 11.18 0 0 0 11.91 3 0.33 0 0 0.33 DAL 3DW Josh Green
51 2439514 Immanuel Quickley 31.30 1 4 13 4 4 2 9 2 4 4 5 3 0 4 14 0 1 0 0 23.28 0 0 0 35.07 19 0.31 1 0.22 0.50 NYK SHW Immanuel Quickley
52 3005116 Quentin Grimes 16.20 1 5 9 0 0 3 6 2 3 1 2 0 0 1 13 1 4 0 0 23.83 0 0 0 17.89 9 0.56 0 0.50 0.67 NYK 3DW Quentin Grimes
53 3005227 Isaiah Jackson 33.90 1 5 12 5 8 0 1 5 11 7 1 3 0 0 15 3 5 0 0 18.63 0 0 0 34.03 13 0.42 0.63 0 0.45 IND VF Isaiah Jackson
54 3005228 Chris Duarte 48.90 1 10 16 5 5 2 3 8 13 7 3 3 0 0 27 2 3 0 0 38.53 0 0 0 19.92 16.50 0.63 1 0.67 0.62 IND SCW Chris Duarte
55 3005417 Herb Jones 28.80 1 5 7 0 0 1 3 4 4 4 4 3 0 2 11 1 3 0 0 37.08 0 0 0 11.13 8 0.71 0 0.33 1 NOP 3DW Herb Jones
56 3007111 Jonathan Kuminga 14.50 1 0 0 5 6 0 0 0 0 5 3 0 0 1 5 1 3 1 0 15.37 0 0 0 12.05 3 0 0.83 0 0 GSW VF Jonathan Kuminga
57 3014026 Keifer Sykes 17.30 1 4 12 0 0 2 5 2 7 4 3 0 0 2 10 2 3 0 0 31.02 0 0 0 19.33 12 0.33 0 0.40 0.29 IND DIS Keifer Sykes
58 3015595 Duane Washington 8.50 1 3 7 0 0 2 2 1 5 0 1 0 0 1 8 0 1 0 0 18.37 0 0 0 18.14 8 0.43 0 1 0.20 IND SHW Duane Washington
59 3015773 Jose Alvarado 31.00 1 6 9 0 0 1 2 5 7 0 4 4 0 0 13 0 3 0 0 19.97 0 0 0 20.67 9 0.67 0 0.50 0.71 NOP DIS Jose Alvarado
蟒蛇。 试图编写一个称为one_frame的函数。 似乎不起作用。 帮助将不胜感激
[英]Python. Trying to write a function called one_frame. Does not seem to work. Help would be greatly appreciated
第一年的项目：我的 while 循环中的一个小错误让我发疯。 非常感谢任何帮助
[英]A first-year project: A small error in my while loop is driving me CRAZY. Any help greatly appreciated
接收值错误：x 和 y 的 x 和 y 值的大小必须相同。 任何帮助，将不胜感激
[英]Receiving ValueError: x and y must be the same size for x and y values. Any help would be appreciated
作为刚开始学习 cs 的初学者，对 python 概念感到困惑。 任何帮助将非常感激
[英]Confused over a python concept as a beginner who has only started learning cs. Any help would be really appreciated
获取已定义列表对象的名称错误。应感谢任何可能的帮助，
[英]getting name error for already defined list objects.any possible help should be appreciated,
如何使用python中的beautifulsoup从“span”标签中使用“data-reactid”进行网络抓取？
[英]How do you webscrape data from a “span” tag with a “data-reactid” using beautifulsoup in python?
试图抓取“hellopeter.com”无法深入了解实际评论
[英]Trying to webscrape 'hellopeter.com' can't dig into the actual review
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.