BeaufifulSoup, lxml navigation for tag with “-” in the tag name?
Question
<body>
<response status="success">
<policy>
<shared/>
<panorama>
<address>
<entry name="text">
<tag1></tag1>>
<tag2></tag2>
</entry>>
</address>
<service>
....
</service>
<pre-rulebase>
</pre-rulebase>
<security>
<rules>
<entry name="some text">
<tag1>text</tag1>>
<tag2>text</tag2>
</entry>
<entry name="more text">
<tag1>text</tag1>
<tag2>text</tag2>
</entry>
...
</rules>
</security>
<post-rulebase>
<entry name="some text">
<tag1>text</tag1>>
<tag2>text</tag2>
</entry>
<entry name="more text">
<tag1>text</tag1>>
<tag2>text</tag2>
</entry>
</post-rulebase>
</panorama>
</policy>
</response>
</body>
Hi,
I am trying to parse above xml file using Python BeautifulSoup and lxml. Usually I navigate to the element using '.'. eg
from bs4 import BeautifulSoup
with open('sample.xml', 'r') as xml_file:
soup = BeautifulSoup(xml_file, 'lxml')
for item in soup.body.response.policy.panorama.address.find('entry'):
<some code action>
My problem is with navigating via above for tags like '' and ''. Since there is "-" in the tag name, the "." navigation is not working. Also since the child tags have same names, i cannot use it direct find. How can I navigate and iterate thru tags under '' ie '' tags?
2 answers
solution1
2 2020-06-17 18:47:34
You can probably do it like this:
from lxml import etree
rules = """[your xml, fixed]"""
doc = etree.XML(rules)
for i in doc.xpath('//post-rulebase//entry'):
print(i.tag,i.attrib['name'])
for t in i.xpath('.//*'):
print(t.tag,t.text)
Output:
entry some text
tag1 text
tag2 text
entry more text
tag1 text
tag2 text
solution2
0 2020-06-17 21:45:57
Another method.
from simplified_scrapy import SimplifiedDoc, req, utils
html = '''
<address>
<entry name="text">
<tag1></tag1>>
<tag2></tag2>
</entry>>
</address>
<service>
....
</service>
<post-rulebase>
<entry name="some text">
<tag1>text</tag1>>
<tag2>text</tag2>
</entry>
<entry name="more text">
<tag1>text</tag1>>
<tag2>text</tag2>
</entry>
</post-rulebase>'''
doc = SimplifiedDoc(html)
entry = doc.select('post-rulebase').entry
print(entry)
print(entry.children.text)
Result:
{'name': 'some text', 'tag': 'entry', 'html': '\n <tag1>text</tag1>>\n <tag2>text</tag2>\n '}
['text', 'text']
Here are more examples: https://github.com/yiyedata/simplified-scrapy-demo/tree/master/doc_examples
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.