"使用 WebKit 和 Selenium 工具抓取 Javascript 网站"

Question

I tried scraping a javascript website using two tools, both didn't work. The website link is: http:\/\/xx.xxx.com\/category-499399872.htm<\/a> The relevant text I'm trying to extract is GY-68...<\/strong> :

<div class="item3line1">

    <dl class="item " data-id="38952795780">
        <dt class="photo">
            <a target="_blank" href="//item.xxx.com/item.htm?spm=a1z10.5-c.w4002-6778075404.11.54MDOI&id=38952795780" data-spm-wangpu-module-id="4002-6778075404" data-spm-anchor-id="a1z10.5-c.w4002-6778075404.11">
                <img src="//img.xxx.com/bao/uploaded/i4/TB1HMt3FFXXXXaFaVXXXXXXXXXX_!!0-item_pic.jpg_240x240.jpg" alt="GY-68 BMP180 新款 BOSCH温度 气压传感器模块 代替BMP085"></img>
            </a>
        </dt>

Answer 1

也许这是一个愚蠢的建议，但是您正在尝试通过类名称“ col-main”查找元素，而示例代码的类名称为“ item-name”。

Answer 2

There is a space in the class name it is 'item ' not 'item'.For that you have to rewrite the xpath as

  //dl[@class="item "]/dt[@class="photo"]/a/img

There is an option to override that.You can use normalize-space() function which strips leading and trailing white-space from a string.

  //dl[normalize-space(@class)="item"]/dt[@class="photo"]/a/img

Or you can go with

  //a[@class='item-name']

also refers to the element and the text is equal to the img's alt attribute

Answer 3

You must check these scraping websites . These are best scraping tools and I am using them.