获取特定标签 - BeautifulSoup

Question

Below is the xml that I'm trying to parse.

<url>
   <loc>https://www.houseofindya.com/aqua-chanderi-pleated-sharara-pants-177/iprdt</loc>
   <image:image>
      <image:loc>https://img.faballey.com/Images/Product/IPL00325Z/d3.jpg</image:loc>
      <image:title>Green Chanderi Pleated Sharara Pants</image:title>
   </image:image>
   <priority>0.8</priority>
   <changefreq>daily</changefreq>
</url>
<url>
   <loc>https://www.houseofindya.com/aqua-foil-chanderi-kurta-171/iprdt</loc>
   <image:image>
      <image:loc>https://img.faballey.com/Images/Product/ITN01710Z/d3.jpg</image:loc>
      <image:title>Aqua Foil Chanderi Kurta</image:title>
   </image:image>
   <priority>0.8</priority>
   <changefreq>daily</changefreq>
</url>

I need to get text of only <loc> tags. So, I do the following:-

soup = BeautifulSoup(xml, features='xml')
loc = soup.find('loc')
while loc is not None:
    url = loc.text
    yield url
    loc = loc.find_next('loc')

The result I get is

1. https://www.houseofindya.com/aqua-chanderi-pleated-sharara-pants-177/iprdt
2. https://img.faballey.com/Images/Product/IPL00325Z/d3.jpg
3. https://www.houseofindya.com/aqua-foil-chanderi-kurta-171/iprdt
4. https://img.faballey.com/Images/Product/ITN01710Z/d3.jpg

However, what I want is only https://www.houseofindya.com/aqua-chanderi-pleated-sharara-pants-177/iprdt , and https://www.houseofindya.com/aqua-foil-chanderi-kurta-171/iprdt . I don't want the text of <image:loc> .

What am I missing here?

Answer 1

You can use CSS selectors even in XML document, so select all url > loc :

from bs4 import BeautifulSoup

xml_doc = """
... your XML from question here ...
"""

soup = BeautifulSoup(xml_doc, "html.parser")

for loc in soup.select("url > loc"):
    print(loc.text)

Prints:

https://www.houseofindya.com/aqua-chanderi-pleated-sharara-pants-177/iprdt
https://www.houseofindya.com/aqua-foil-chanderi-kurta-171/iprdt