简体   繁体   中英

Get table from web page

I am trying to get table from web page using Selenium webdriver. First I am logging in to webpage, then proceeding to web page with table.

The problem is that table does not have td tags and web page source code looks like:

在此处输入图片说明

C# code:

using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using System;
using System.Collections.Generic;

namespace Table_Scrape
{
    class Program
    {
        static void Main(string[] args)
        {

            IWebDriver driver = new ChromeDriver();
            driver.Navigate().GoToUrl("https://website.com/login");
            driver.FindElement(By.Id("username")).SendKeys("MYusername");
            driver.FindElement(By.Id("password")).SendKeys("MYpassword");
            driver.FindElement(By.Id("btnSubmit_6")).Click();

            driver.FindElement(By.Id("btnContinue")).Click();

            driver.Navigate().GoToUrl("https://website.com/table");

            IList<IWebElement> allElement = driver.FindElements(By.TagName("td"));
            foreach (IWebElement element in allElement)
            {
                string cellText = element.Text;
                Console.WriteLine(cellText);
            }

        }
    }
}

Does anyone have any suggestions on how to get data out of webpage?

Table structure is the same for each row. Columns are like slick-cell 10 r0 , slick-cell 10 r1 etc.


EDIT:

Console output:

DevTools listening on ws://xxxx:xxx/devtools/browser/0a02f6b7-3c33-41ea-b0b3-fb67d3f436c7
[1583946337.939][WARNING]: Timed out connecting to Chrome, retrying...
[1583946340.587][SEVERE]: Timed out receiving message from renderer: 0.100
[1583946343.800][SEVERE]: Timed out receiving message from renderer: 0.100
[1583946343.903][SEVERE]: Timed out receiving message from renderer: 0.100
[1583946344.024][SEVERE]: Timed out receiving message from renderer: 0.100
[1583946345.482][SEVERE]: Timed out receiving message from renderer: 0.100
[1583946345.583][SEVERE]: Timed out receiving message from renderer: 0.100
[1583946347.170][SEVERE]: Timed out receiving message from renderer: 0.100

You need to loop through all the elements that have the "slick-cell" class. Here is a link to the code to do that: https://stackoverflow.com/a/31606955/10880378

Instead of IList<IWebElement> allElement = driver.FindElements(By.TagName("td")); I would use IList<IWebElement> allElement = driver.FindElements(By.Xapth("//div[contains(@class,'slick-cell')]")); However if this class name is not unique then different approach would be needed. If that's the case. let me know and we probably will need to do it via parent div>child div depending open which table you want to iterate.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM