lxml etree get all text before element

Question

How to get all text before an element in a etree separated from the text after the element?

from lxml import etree

tree = etree.fromstring('''
    <a>
        find
        <b>
            the
        </b>
        text
        <dd></dd>
        <c>
            before
        </c>
        <dd></dd>
        and after
    </a>
''')

What do I want? In this example, the <dd> tags are separators and for all of them

for el in tree.findall('.//dd'):

I would like to have all text before and after them:

[
    {
        el : <Element dd at 0xsomedistinctadress>,
        before : 'find the text',
        after : 'before and after'
    },
    {
        el : <Element dd at 0xsomeotherdistinctadress>,
        before : 'find the text before',
        after : 'and after'
    }
]

My idea was to use some kind of placeholders in the tree with which I replace the <dd> tags and then cut the string at that placeholder, but I need the correspondence with the actual element.

Answer 1

There might be a simpler way, but I would use the following XPath expressions:

preceding-sibling::*/text()|preceding::text()
following-sibling::*/text()|following::text()

Sample implementation (definitely violating the DRY principle):

def get_text_before(element):
    for item in element.xpath("preceding-sibling::*/text()|preceding-sibling::text()"):
        item = item.strip()
        if item:
            yield item

def get_text_after(element):
    for item in element.xpath("following-sibling::*/text()|following-sibling::text()"):
        item = item.strip()
        if item:
            yield item

for el in tree.findall('.//dd'):
    before = " ".join(get_text_before(el))
    after = " ".join(get_text_after(el))

    print {
        "el": el,
        "before": before,
        "after": after
    }

Prints:

{'el': <Element dd at 0x10af81488>, 'after': 'before and after', 'before': 'find the text'}
{'el': <Element dd at 0x10af81200>, 'after': 'and after', 'before': 'find the text before'}