使用lxml，如何读取嵌套元素中的文本？

Question

I'm trying to search about 500 XML documents for some specific phrases, and output the ID of any element that contains any of those phrases. Currently, this is my code:

from lxml import etree
import os
import re

files = os.listdir('C:/Users/Me/Desktop/xml')
search_words = ['House divided', 'Committee divided', 'on Division', 'Division List',
                'The Ayes and the Noes',]

for f in files:
    doc = etree.parse('C:/Users/Me/Desktop/xml/' +f)
    for elem in doc.iter():
        for word in search_words:
            if elem.text is not None and str(elem.attrib) != "{}" and word in elem.text and len(re.findall(r'\d+', elem.text))>1:
                votes = re.findall(r'\d+', elem.text)
                string = str(elem.attrib)[8:-2] + ","
                string += (str(votes[0]) + "," + str(votes[1]) + ",")
                string += word + ","
                string += str(elem.sourceline)
                print string      

Input like this will output properly:

<p id="S3V0001P0-01869">The House divided; Against the Motion 83; For it 23&#x2014;Majority 60.</p>

But input with nested elements like this will be missed, because the text inside is not being parsed for the phrases:

<p id="S3V0141P0-01248"><member>THE CHANCELLOR OF THE EXCHEQUER</member><membercontribution> said, that the precedent occurred on the 8th of April, 1850, on a Motion for going into a Committee of Supply. An Amendment was moved by Captain Boldero on the subject of assistant-surgeons in the navy, when, on a division being called for, the Question was put that the words proposed to be left out stand part of the Question. The House divided, when the numbers were&#x2014;Ayes, 40; Noes, 48. The Question, "That the proposed words be added" was put and agreed to; the main Question, as amended, was put and agreed to; and the Question being then put, "That Mr. Speaker do now leave the chair," that Motion was agreed to, and the House went into Committee of Supply.</membercontribution></p>

Is there any way to read the text inside nested elements like this and return its ID?

Answer 1

With lxml there is an xpath method and XPath has a contains function you can use with eg

doc = ET.fromstring('<p id="S3V0141P0-01248"><member>THE CHANCELLOR OF THE EXCHEQUER</member><membercontribution> said, that the precedent occurred on the 8th of April, 1850, on a Motion for going into a Committee of Supply. An Amendment was moved by Captain Boldero on the subject of assistant-surgeons in the navy, when, on a division being called for, the Question was put that the words proposed to be left out stand part of the Question. The House divided, when the numbers were&#x2014;Ayes, 40; Noes, 48. The Question, "That the proposed words be added" was put and agreed to; the main Question, as amended, was put and agreed to; and the Question being then put, "That Mr. Speaker do now leave the chair," that Motion was agreed to, and the House went into Committee of Supply.</membercontribution></p>')
result = doc.xpath('//*[@id and contains(., $word)]', word = 'House divided')

Answer 2

You could use some XPath and extract all the text elements below whatever's interesting. I like Parsel : pip install parsel .

import parsel

data = ('<x><y><z><p id="S3V0141P0-01248"><member>THE CHANCELLOR OF THE EXCHEQUER'
        '</member><membercontribution> said, that the precedent occurred on the '
        '8th of April, 1850, on a Motion ...</membercontribution></p></z></y></x>')

selector = parsel.Selector(data)

for para in selector.xpath('//p'):
    id = para.xpath('@id').extract_first()
    texts = para.xpath('*/text()').extract()
    for text in texts:
        # do whatever search
        print(id, len(text), 'April' in text)

Output:

S3V0141P0-01248 31 False
S3V0141P0-01248 77 True