Python xml.dom.minidom generates invalid XML?

Question

I have encountered strange problem with xml.dom.minidom python package. I generate a document, populating it with data taken from terminal. Sometimes such data contain terminal control characters. When I stored such character in text data node with minidom.toprettyxml() everything seems to be fine, however, the generated document is not a valid XML.

Does anyone know why minidom allows to generate invalid document? Is this connected with "mini" part?

Here is the extracted example code (with some system info too):

Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from xml.dom import minidom
>>> impl = minidom.getDOMImplementation()
>>> doc = impl.createDocument(None, "results", None)
>>> root = doc.firstChild
>>> outString = "test "+chr(1) #here goes control character
>>> root.appendChild(doc.createTextNode(outString))
<DOM Text node "'test \x01'">
>>> doc.toprettyxml(encoding="utf-8")
'<?xml version="1.0" encoding="utf-8"?>\n<results>\n\ttest \x01\n</results>\n'
>>> with open("/tmp/outfile", "w") as f:
...     f.write(doc.toprettyxml(encoding="utf-8"))
... 
>>> doc2 = minidom.parse("/tmp/outfile")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/xml/dom/minidom.py", line 1918, in parse
    return expatbuilder.parse(file)
  File "/usr/lib/python2.6/xml/dom/expatbuilder.py", line 924, in parse
    result = builder.parseFile(fp)
  File "/usr/lib/python2.6/xml/dom/expatbuilder.py", line 207, in parseFile
    parser.Parse(buffer, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 3, column 6
>>> open("/tmp/outfile","r").readlines()
['<?xml version="1.0" encoding="utf-8"?>\n', '<results>\n', '\ttest \x01\n', '</results>\n']
>>> 

Answer 1

Looking at the code for _write_data it only escapes ampersands, slashes and brackets:

def _write_data(writer, data):
    "Writes datachars to writer."
    data = data.replace("&", "&amp;").replace("<", "&lt;")
    data = data.replace("\"", "&quot;").replace(">", "&gt;")
    writer.write(data)

As you surmised, minidom isn't intended as a fully robust implementation (its implementation of namespaces is lacking, for instance).