Beautiful Soup AttributeError: 'NoneType' object has no attribute 'find_all' 即使网页结构相同
[英]Beautiful Soup AttributeError: 'NoneType' object has no attribute 'find_all' even though webpage is structured the same
问题描述
so I have working code for one web-scraper and want to use it on another website which is structured the same.
所以我有一个网络爬虫的工作代码，并想在另一个结构相同的网站上使用它。
The code I have that works is:我拥有的代码是：
url = "https://efl.network/index/efl/LeaguePassingStats.html"
page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
tb = soup.find('table', class_='tablesorter')
table_rows = tb.find_all("tr")
however if i change the url to但是，如果我将url更改为
https://sim-football.com/indexes/DSFLS22/LeaguePassingStats.html
it gives me the error它给了我错误
table_rows = tb.find_all("tr")
AttributeError: 'NoneType' object has no attribute 'find_all'
Both websites seem to be structured the same however.然而，这两个网站的结构似乎相同。
2 个解决方案
解决方案1
0 2020-05-02 00:33:09
This line returns None此行返回无
tb = soup.find('table', class_='tablesorter')
That means, there is no table element with class 'tablesorter' on the page.这意味着，页面上没有带有 class 'tablesorter' 的表格元素。
解决方案2
0 2020-05-02 00:38:39
If you print out page.content you can see this is the HTML that is returned: '<head><title>Not Acceptable.</title></head><body><h1>Not Acceptable.</h1><p>An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.</p></body></html>'如果您打印出page.content ，您可以看到这是返回的 HTML： '<head><title>Not Acceptable.</title></head><body><h1>Not Acceptable.</h1><p>An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.</p></body></html>' '<head><title>Not Acceptable.</title></head><body><h1>Not Acceptable.</h1><p>An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.</p></body></html>'
So the website is blocking your web scrape for some reason and returning this error HTML, in which the table element you're looking for doesn't exist, causing your error因此，该网站出于某种原因阻止了您的 web 抓取并返回此错误 HTML，其中您要查找的table元素不存在，导致您的错误
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.