Scrapy 从 HTML 中提取内容没有 output

Question

所以基本上我想从这个 html 拉出 tr-mfgPartNumber class 下的零件但是有问题。

• 首先我认为这是我调用每个 class 但仍然没有 output 的语法

• 尝试将另一个 for 循环添加到 go 到整个身体 class 如果有人可以检查我的代码是否在我调用类的方式上有错误，那就太好了！

   import scrapy class DigiSpider(scrapy.Spider): name = 'digi' allowed_domains = ['digikey.com'] start_urls = ['https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-1%7C428%2C-8%7C774%2C7%7C1/'] def parse(self, response): data={} parts=response.css('tbody.InkPart') for part in parts: for p in part.css('td.tr-mfgPartNumber'): data['href'] = p.css('a::attr(href)').extract() yield data

下面是 HTML

<tbody id="lnkPart" cookie-tracking="ref_page_event=Select Part;available_parameters=[&quot;s&quot;,&quot;pv1989&quot;,&quot;pv142&quot;,&quot;pv2042&quot;,&quot;pv2192&quot;,&quot;pv276&quot;,&quot;pv252&quot;,&quot;pv16&quot;,&quot;pv1291&quot;];">

<tr>
    
    <td class="tr-compareParts" align="center">
        <input type="checkbox" name="part" value="428-3574-2-ND" id="428-3574-2-ND" onclick="partClick();">
        <label title="Compare Parts" for="428-3574-2-ND"></label>
    </td>
    
    <td class="tr-datasheet">
            <a class="lnkDatasheet" href="https://www.cypress.com/file/43021/download" target="_blank" track-data="ref_page_event=Display Asset;page_title=Datasheet;asset_type=Datasheet">
                <img class="datasheet-img" src="//www.digikey.com/Web%20Export/Common/icons/datasheet.png" alt="CY62157EV30LL-45ZSXIT Datasheet" title="CY62157EV30LL-45ZSXIT Datasheet">
            </a>
    </td>
    
    <td class="tr-image">
        <a href="/product-detail/en/cypress-semiconductor-corp/CY62157EV30LL-45ZSXIT/428-3574-2-ND/1205268">
            <img class="pszoomer" zoomimg="//media.digikey.com/Renders/Cypress%20Semi%20Renders/428;51-85087;Z,ZS;44.jpg" border="0" height="64" src="//media.digikey.com/Renders/Cypress%20Semi%20Renders/428;51-85087;Z,ZS;44_tmb.jpg" alt="CY62157EV30LL-45ZSXIT - Cypress Semiconductor Corp" title="CY62157EV30LL-45ZSXIT - Cypress Semiconductor Corp">
        </a>
    </td>
    
    <td class="tr-dkPartNumber nowrap-culture">                             
        <a href="/product-detail/en/cypress-semiconductor-corp/CY62157EV30LL-45ZSXIT/428-3574-2-ND/1205268">
            428-3574-2-ND
        </a>
            <div class="product-indicator-collection">
<a class="align-indicator-collection" href="javascript:msgBox('#dlgRohs');">
<img class="rohs-foilage" src="//www.digikey.com/web%20export/common/mkt/en/leaf.png" border="0" alt="This part is RoHS compliant." title="This part is RoHS compliant.">
</a>


</div>
       
    </td>
    
    <td class="tr-mfgPartNumber">
        <a href="/product-detail/en/cypress-semiconductor-corp/CY62157EV30LL-45ZSXIT/428-3574-2-ND/1205268">
            <span>CY62157EV30LL-45ZSXIT</span>
        </a>
    </td>

Answer 1

当我尝试相同的代码时，scrapy 得到空响应。 也许该站点正在检测并阻止蜘蛛。 使用用户代理后，它工作了。

这是下面的代码（我也将“tbody.InkPart”更改为“tbody#lnkPart”，这是您的代码中的语法错误，尽管它不是必需的，因为只有一个 tbody 标签）：

import scrapy


class DigiSpider(scrapy.Spider):
    name = 'digi'
    allowed_domains = ['digikey.com']
    custom_settings = {
        "USER_AGENT": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36"
    }
    start_urls = ['https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-1%7C428%2C-8%7C774%2C7%7C1/']

    def parse(self, response):
        data={}
        parts=response.css('tbody#lnkPart')
        for part in parts:
            for p in part.css('td.tr-mfgPartNumber'):
                data['href'] = p.css('a::attr(href)').extract()
                yield data