I can't get full html code when souping a dynamic table using Beautiful Soup/Selenium
Question
I am having troubles souping a dynamic table with Beautiful soup/Selenium
When I run the code, I can´t get the entire html code in my notebook.
The tags I am interested with are under the tag 'frameset' which I should somehow "open". How could I do this?
driver = webdriver.Chrome('/Users/Administrador/Documents/chromedriver')
main_url = 'https://www.justiciacordoba.gob.ar/Estatico/JEL/Escrutinios/ReportesEleccion20190512/default.html'
driver.get(main_url)
soup = BeautifulSoup(driver.page_source, 'lxml')
soup
output : <html><head></head><frameset border="0" cols="*" frameborder="NO" framespacing="0" id="fset" rows="190,*">
<frame name="topFrame" scrolling="NO" src="Index.html"/>
<frame align="center" name="mainFrame" src="about:blank"/>
</frameset><noframes></noframes>
</html>
1 answers
solution1
1 ACCPTED 2020-05-11 19:17:01
Change the driver's frame and store the value:
driver.switch_to.frame(driver.find_element_by_name("topFrame"))
frame_source = driver.page_source
The frame_source variable should store the iframe's source code. I checked the website, and the iframe's name is 'topFrame'.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.
Related Question
I can't get all of image attributes using Beautiful Soup and Selenium
Can't find a table using Beautiful soup
Beautiful Soup won't return full table in from HTML object
selenium or beautiful soup to get data from dynamic table cell
Get Dynamic Tabular from Website data using Selenium & Beautiful Soup
Using Beautiful Soup to get the full URL in source code
Full Html code not importing Beautiful Soup
Beautiful Soup not parsing full websites HTML code
How to get the full html with Beautiful Soup?
I'm not able to scrape table data using selenium and beautiful soup
Related Tags