简体   繁体   中英

Xpath for loop gives only first result. How do I get all of them? (Python, Scrapy)

I'm trying to get all results that have specific class name in them. Xpath for loop only gives the first result.

Heres the code:

for links in response.xpath("//*[@id='main']/div/div[contains(@class, 'ZINbbc xpd O9g5cc uUPGi')]"):
        link = links.xpath("//div[contains(@class, 'kCrYT')]/a/@href").get()

        header = links.xpath("//div[contains(@class, 'kCrYT')]/a/h3/div/text()").get()

I've tried adding [position()>=3 and position(<=14] to the first div too.

Edit: Heres the HTML:

<div id="main">
    <div>...</div>(Don't need this one)
    <styles>...</styles>
    <div>...</div>
    <div class="ZINbbc xpd O9g5cc uUPGi">
        <div class="kCrYT">
            <a href="link">
                <h3 class="zBAuLc">
                    <div class="BNeawe vvjwJb AP7Wnd">Title</div>
                </h3>
.......

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM