我试图从espn中提取一些数据作为一个表并将其作为列表

Question

start_urls = http://www.espncricinfo.com/series/18679/scorecard/1144998/australia-vs-india-2nd-odi-india-in-aus-2018-19

我抓取了这个网站并提取了比赛结果（获胜团队），然后我产生了玩家网址，我想要打印玩家名称和击球风格。 我的第一个问题是1.我无法抽象玩家击球哨。 它位于<pclass="ciPlayerinformationtxt"><b>Batting style</b> <span>Right-hand bat</span> 。 我只能提取文字'击球风格'。如何提取'右手球'2.我无法将整个提取的数据作为一个表格。 我得到的结果是

所有玩家的链接http://www.espncricinfo.com/ci/content/player/326434.html

Player_name国家Alex Carey澳大利亚
Kuldeep Yadav印度
Mohammed Siraj印度
WINNING_TEAM：印度

class ScoreSpider(scrapy.Spider):
    name = 'score'
    allowed_domains = ['espncricinfo.com']

    def parse(self, response):
        Player_URLs=[]

        #got the result

        result= response.xpath('//div[@class="cscore_notes"]/span/text()').extract_first()
        result=result.split(" ")
        Winning_Team =result[0]

        #extracted player ulrs

        Batting_Player_URLs=response.xpath('//div[@class="cell batsmen"]/a/@href').extract()
        Bowling_Player_URLs=response.xpath('//*[@class="scorecard-section bowling"]/table/tbody/tr/td/a/@href').extract()

        #added to a list

        Player_URLs.extend(Batting_Player_URLs)
        Player_URLs.extend(Bowling_Player_URLs)
        for p in Player_URLs:
            yield Request(p,callback=self.parse_players,meta={'p':p})
            yield{'Winning_Team':Winning_Team}

    def parse_players(self,response):
        Player_name=response.xpath('//div[@class="ciPlayernametxt"]/div/h1/text()').extract_first()
        Country=response.xpath('//div[@class="ciPlayernametxt"]/div/h3/b/text()').extract_first()

        #this wont give the batting style but the 'batting style' as text
        Batting_style=response.xpath('//div[@class="ciPlayerinformationtxt"]/p/text()').extract_first()

        yield{'Player_name':Player_name,
              'Country':Country, 
              'Batting_style':Batting_style}

我想要的是将提取的数据作为单个表格，我想避免重复。

          yield{'Winning_Team':Winning_Team,
                'Player_name':Player_name,
                'Country':Country,
                'Batting_style':Batting_style}

提前致谢

Answer 1

您需要调整XPath：

batting_style = response.xpath('//p[@class="ciPlayerinformationtxt"]/b[.="Batting style"]/following-sibling::span[1]/text()').get()

UPDATE

for p in Player_URLs:
    yield Request(p,callback=self.parse_players,meta={'Winning_Team':Winning_Team})

然后：

def parse_players(self,response):
    Winning_Team = response.meta["Winning_Team"]