data from dynamically loading webpage

Question

I am trying to get data from the following website:

http://www.mcxindia.com/SitePages/RealTimeData.aspx

But the source file has only JS codes, I need the financial data, and I am fooling around selenium to get them. But still I could not locate the class name in selenium but the class name is appearing when I am doing "Inspect Element". Below is the code:

 from selenium.webdriver.support.ui import WebDriverWait
 from selenium.webdriver.support import expected_conditions as EC
 from selenium.webdriver.common.by import By
 from selenium import webdriver

 # Start the WebDriver and load the page
 wd = webdriver.Firefox()
 wd.get("http://www.mcxindia.com/SitePages/RealTimeData.aspx")

 # Wait for the dynamically loaded elements to show up
 WebDriverWait(wd, 30).until(
 EC.visibility_of_element_located((By.CLASS_NAME, "tablerow")))

 # And grab the page HTML source
 html_page = wd.page_source
 print html_page
 wd.quit()

Help would be greatly appreciated!

Answer 1

That table sits inside an iframe. Look through the documentation on your language binding of Selenium for instructions how to switch to a new frame.

In Java, for example:

driver.switchTo().frame("name or ID");

You'll need to look through the DOM on your page to grab the required locator of the root of the iframe, but it won't be difficult.

Also, to note... if you will be performing any other actions after you complete any steps inside your iframe, you'll need to switch back to the main page before moving on.