使用xpath和scrapy从HTML提取特定值

Question

I have following html Code:

 <tr data-live="COumykPG" data-dt="10,11,2017,19,00" data-def="1"> <td class="table-matches__tt"><span class="table-matches__time" data-live-cell="time">19:00</span><a href="/soccer/germany/oberliga-bremen/oberneuland-habenhauser/COumykPG/" data-live-cell="matchlink"><span>Oberneuland</span> - <span>Habenhauser</span></a></td> <td class="livebet" data-live-cell="livebet">&nbsp;</td> <td class="table-matches__streams" data-live-cell="score"> </td> <td class="table-matches__odds" data-oid="2p2k5xv464x0x6ev9v"><a href="/myselections.php?action=3&amp;matchid=COumykPG&amp;outcomeid=2p2k5xv464x0x6ev9v&amp;otheroutcomes=2p2k5xv498x0x0,2p2k5xv464x0x6eva0" onclick="return my_selections_click('1x2', 'soccer');" title="Add to My Selections" target="mySelections">1.10</a></td> <td class="table-matches__odds" data-oid="2p2k5xv498x0x0"><a href="/myselections.php?action=3&amp;matchid=COumykPG&amp;outcomeid=2p2k5xv498x0x0&amp;otheroutcomes=2p2k5xv464x0x6ev9v,2p2k5xv464x0x6eva0" onclick="return my_selections_click('1x2', 'soccer');" title="Add to My Selections" target="mySelections">7.44</a></td> <td class="table-matches__odds" data-oid="2p2k5xv464x0x6eva0"><a href="/myselections.php?action=3&amp;matchid=COumykPG&amp;outcomeid=2p2k5xv464x0x6eva0&amp;otheroutcomes=2p2k5xv464x0x6ev9v,2p2k5xv498x0x0" onclick="return my_selections_click('1x2', 'soccer');" title="Add to My Selections" target="mySelections">12.40</a></td> </tr> 

I try to scrap from the following code the 3 float values: 1,10 7.44 12.40 The expression that i tried to use for geting the value was the following:

response.xpath('//a/@target').extract()

Output that I get is 'mySelections' .

Iwant to get the value next to it. What is the right expression for it?

Thank you in advance

Answer 1

What's wrong

response.xpath('//a/ @target ').extract()

Why?

• If you format your HTML, the error is obvious.

  You want to extract text from a tag, not the target attribute.

    <tr data-live="COumykPG" data-dt="10,11,2017,19,00" data-def="1"> <td class="table-matches__tt"> <span class="table-matches__time" data-live-cell="time">19:00</span> <a href="/soccer/germany/oberliga-bremen/oberneuland-habenhauser/COumykPG/" data-live-cell="matchlink"> <span>Oberneuland</span> - <span>Habenhauser</span> </a> </td> <td class="livebet" data-live-cell="livebet">&nbsp;</td> <td class="table-matches__streams" data-live-cell="score"></td> <td class="table-matches__odds" data-oid="2p2k5xv464x0x6ev9v"> <a href="/myselections.php?action=3&amp;matchid=COumykPG&amp;outcomeid=2p2k5xv464x0x6ev9v&amp;otheroutcomes=2p2k5xv498x0x0,2p2k5xv464x0x6eva0" onclick="return my_selections_click('1x2', 'soccer');" title="Add to My Selections" target="mySelections">1.10</a> </td> <td class="table-matches__odds" data-oid="2p2k5xv498x0x0"> <a href="/myselections.php?action=3&amp;matchid=COumykPG&amp;outcomeid=2p2k5xv498x0x0&amp;otheroutcomes=2p2k5xv464x0x6ev9v,2p2k5xv464x0x6eva0" onclick="return my_selections_click('1x2', 'soccer');" title="Add to My Selections" target="mySelections">7.44</a> </td> <td class="table-matches__odds" data-oid="2p2k5xv464x0x6eva0"> <a href="/myselections.php?action=3&amp;matchid=COumykPG&amp;outcomeid=2p2k5xv464x0x6eva0&amp;otheroutcomes=2p2k5xv464x0x6ev9v,2p2k5xv498x0x0" onclick="return my_selections_click('1x2', 'soccer');" title="Add to My Selections" target="mySelections">12.40</a> </td> </tr> 

  How to fix it

• Use one of those followings

  • response.xpath('//a/text()').extract()

  • According to other developers, response.xpath sometimes will cause bugs, you should use scrapy's selector instead.

     from scrapy.selector import Selector result_array = Selector(text=response.body).xpath('//a/text()').extract()