xmltodict.unparse 未正确处理 CDATA

Question

I am trying to use xmltodict to manipulate an XML content as python object, but I am facing an issue to handle properly CDATA. I think I am missing something somewhere, this is my code:

import xmltodict

data = """<node1>
    <node2 id='test'><![CDATA[test]]></node2>
    <node3 id='test'>test</node3>
</node1>"""

data = xmltodict.parse(data,force_cdata=True, encoding='utf-8')
print data

print xmltodict.unparse(data, pretty=True)  

And this is the output:

OrderedDict([(u'node1', OrderedDict([(u'node2', OrderedDict([(u'@id', u'test'), ('#text', u'test')])), (u'node3', OrderedDict([(u'@id', u'test'), ('#text', u'test')]))]))])
<?xml version="1.0" encoding="utf-8"?>
<node1>
        <node2 id="test">test</node2>
        <node3 id="test">test</node3>
</node1>

We can see here that the CDATA is missing in the generated node2, and also node2 is the same as node3. However, in the input the nodes are different.

Regards

Answer 1

I finally managed to get it working by performing this monkey-patch. I am still not very happy with it, It's really a 'hack' this feature should be included somewhere properly:

import xmltodict
def escape_hacked(data, entities={}):
    if data[0] == '<' and  data.strip()[-1] == '>':
        return '<![CDATA[%s]]>' % data

    return escape_orig(data, entities)


xml.sax.saxutils.escape = escape_hacked

and then run your python code normally:

data = """<node1>
    <node2 id='test'><![CDATA[test]]></node2>
    <node3 id='test'>test</node3>
</node1>"""

data = xmltodict.parse(data,force_cdata=True, encoding='utf-8')
print data

print xmltodict.unparse(data, pretty=True) 

To explain, the following line detect if the data is a valid XML, then it add the CDATA tag arround it:

    if data[0] == '<' and  data.strip()[-1] == '>':
        return '<![CDATA[%s]]>' % data

Regards

Answer 2

I want to clarify that there is no officially supported way to keep the CDATA section.

You could check the issue here.

Based on the above facts, you need DIY. There are two approaches：

1. Subclassing XMLGenerator

import xmltodict

class XMLGenerator(xmltodict.XMLGenerator):
    def characters(self, content):
        if content:
            self._finish_pending_start_element()
            if not isinstance(content, str):
                content = str(content, self._encoding)
            self._write('<![CDATA[' + content + ']]>')

xmltodict.XMLGenerator = XMLGenerator

It is not a hack, so it won't change the behaviour of xmltodict other than unparse() . More importantly, it won't pollute the built-in library xml .

out_xml = xmltodict.unparse(data)

From now on, every nodes having character data will keep the CDATA section.

2. Unescaping the escaped XML

from xml.sax.saxutils import unescape

START_CDATA = '<![CDATA['
END_CDATA = ']]>'

def preprocessor(key, value):
    if key in KEEP_CDATA_SECTION:
        if isinstance(value, dict) and '#text' in value:
            value['#text'] = START_CDATA + value['#text'] + END_CDATA
        else:
            value = START_CDATA + value + END_CDATA
    return key, value

Now, you can specify which nodes should be encapsulated into the CDATA section.

KEEP_CDATA_SECTION = ['node2']

out_xml = xmltodict.unparse(data, preprocessor=preprocessor)
out_xml = unescape(out_xml)

However, you shall not try it on the untrusted data, because this approach not only unescapes the character data but also unescapes the nodes' attributes.