在Java中使用XPath和Selenium解析HTML表数据

Question

I want to take the data and organize it without the tags. It looks something like this

<table class="SpecTable">
    <col width="40%" />
    <col width="60%" />
    <tr>
        <td class="LightRowHead">Optical Zoom:</td>
        <td class="LightRow">15x</td>
    </tr>
    <tr>
        <td class="DarkRowHead">Digital Zoom:</td>
        <td class="DarkRow">6x</td>
    </tr>
    <tr>
        <td class="LightRowHead">Battery Type:</td>
        <td class="LightRow">Alkaline</td>
    </tr>
    <tr>
        <td class="DarkRowHead">Resolution Megapixels:</td>
        <td class="DarkRow">14 MP</td>
    </tr>
</table>

and I want to be able to extract all the strings of information so that I can store in a plaintext file with just this:

Optical Zoom: 15x Digital Zoom: 6x Battery Type: Alkaline Resolution Megapixels: 14 MP

public static void main(String[] args) {

        FirefoxProfile profile = new FirefoxProfile();
        profile.setPreference("general.useragent.override", "some UA string");
        WebDriver driver = new FirefoxDriver(profile);

        String Url = "http://www.walmart.com/ip/Generic-14-MP-X400-BK/19863348";
        driver.get(Url);
        List<WebElement> resultsDiv = driver.findElements(By.xpath("//table[contains (@class,'SpecTable')//td"));

        System.out.println(resultsDiv.size());
        for (int i=0; i<resultsDiv.size(); i++) {
            System.out.println(i+1 + ". " + resultsDiv.get(i).getText());
        }

I am programming in Java with Selenium and I cannot figure out the correct XPath expression for it.

Can someone figure out why I err on this and maybe give me some pointers on how I can parse this data correctly? Im very new to Selenium and XPaths but I need this for work.

Also if anyone has any good sources for me to learn Selenium and XPath fast, those would also be greatly appreciated!

Answer 1

The spec is surprisingly a very good read on XPath.

You might also try CSS selectors .

Anyway, one way to get the data from a table can be as following:

// gets all rows
List<WebElement> rows = driver.findElements(By.xpath("//table[@class='SpecTable']//tr"));
// for every line, store both columns
for (WebElement row : rows) {
    WebElement key = row.findElement(By.XPath("./td[1]"));
    doAnythingWithText(key.getText());
    WebElement val = row.findElement(By.XPath("./td[2]"));
    doAnythingWithText(val.getText());
}

Answer 2

Probably this will suite your needs:

string text = driver.findElement(By.cssSelector("table.SpecTable")).getText();

String text will contain all text nodes from the table with class SpecTable. I prefer using css , because it's supported by IE and faster than xpath. But as for xpath tutorials try this and this .

Answer 3

As another option you could grab all the cells of the table into one array and access them that way. EG.

ReadOnlyCollection<IWebElement> Cells = driver.FindElements(By.XPath("//table[@class='SpecTable']//tr//td"));

This will get you all the cells in that table as an array which you can then use to access the text iteratively.

string forOutput = Cells[i].Text;

Answer 4

CSharp method to extract any table in a 2 dimension array:

private string[,] getYourSpecTable(){
    return getArrayBy(By.CssSelector("table.SpecTable tr"), By.CssSelector("td"));
}

private string[,] getArrayBy(By rowsBy, By columnsBy){
    bool init=false;
    int nbRow=0, nbCol=0;
    string[,] ret = null;
    ReadOnlyCollection<OpenQA.Selenium.IWebElement> rows = this.webDriver.FindElements(rowsBy);
    nbRow = rows.Count;
    for(int r=0;r<nbRow;r++) {
        ReadOnlyCollection<OpenQA.Selenium.IWebElement> cols = rows[r].FindElements(columnsBy);
        if(!init) {
            init= true;
            nbCol = cols.Count;
            ret = new string[rows.Count, cols.Count];
        }                
        for(int c=0;c<nbCol;c++) {
            ret[r, c] = cols[c].Text;
        }
    }
    return ret;
}