Python Web Scraping with Beautiful Soup

Question

I'm trying to pull the entire table from the site below and store as a dataframe, but am hitting an error when attempting to pull all the headings. It appears that the table has these attributes, so not sure why this is happening.

URL = "http://www.ercot.com/content/cdr/html/real_time_spp"
page = requests.get(URL).text
soup = BeautifulSoup(page, "lxml")

table = soup.find("table", attrs={"class": "tableStyle"})
table_data = table.tbody.find_all("tr")


---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-241-362ee5fb0444> in <module>
      1 table = soup.find("table", attrs={"class": "tableStyle"})
----> 2 table_data = table.tbody.find_all("tr")

AttributeError: 'NoneType' object has no attribute 'find_all'

Answer 1

The HTML for that page doesn't have a tbody element, which is why table.tbody is None .

You can get all the rows directly from the table using:

table = soup.find("table", attrs={"class": "tableStyle"})
table_data = table.findAll('tr')