NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="events_table"]"}

Question

Need to scrape a table from https://www.ncei.noaa.gov/access/billions/events . There is a link on the website to download the table but downloaded file does not have Summary column. Tried to use following code to scrape the table:

url = 'https://www.ncei.noaa.gov/access/billions/events'
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.implicitly_wait(30)
driver.get(url)
df=pd.read_html(driver.find_element_by_id("events_table").get_attribute('outerHTML'))[0]

but got error

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="events_table"]"}
  (Session info: chrome=102.0.5005.61)

while I manually checked the chrome version on my mac is 100.0.4896.127 .

inspect the table component, found the table id is events_table

Answer 1

Is using find_element_by_id a must or is using find_element_by_xpath fine by you? Because this works if you were to use find_element_by_xpath and provide the id:

df=pd.read_html(driver.find_element_by_xpath('//*[@id="events-table"]').get_attribute('outerHTML'))[0]

Result:

                                                 Event  ... Deaths
0                                Western Wildfires2021  ...      8
1                   Western Drought and Heat Wave2021†  ...    229
2    Midwest Derecho and Tornado OutbreakDecember 2...  ...      1
3    Southeast, Central Tornado OutbreakDecember 2021†  ...     93
4                    Hurricane NicholasSeptember 2021†  ...      0
..                                                 ...  ...    ...
318  Severe Storms, Flash Floods, Hail, TornadoesMa...  ...     20
319                         Florida FreezeJanuary 1981  ...      0
320  Central/ Eastern Drought/ Heat WaveSummer-Fall...  ...   1260
321                         Hurricane AllenAugust 1980  ...     13
322      Southern Severe Storms and FloodingApril 1980  ...      7