在Python /元素树中从300MG Xml中删除元素

Question

I'm trying to parse a 300MB XML in ElementTree, based on advise like Can Python xml ElementTree parse a very large xml file?

from xml.etree import ElementTree as Et

for event, elem in Et.iterparse('C:\...path...\desc2015.xml'):  
    if elem.tag == 'DescriptorRecord':
        for e in elem._children:
            if str(e.tag) in ['DateCreated', 'Year', 'Month', 'TreeNumber', 'HistoryNote', 'PreviousIndexing']:
                e.clear()
                elem.remove(e)
                print 'removed %s' % e

giving...

removed <Element 'HistoryNote' at 0x557cc7f0>
removed <Element 'DateCreated' at 0x557fa990>
removed <Element 'HistoryNote' at 0x55809af0>
removed <Element 'DateCreated' at 0x5580f5d0>

However, this just keeps going, the file isn't getting any smaller, and on inspection the elements are still there. Tried either e.clear() or elem.remove(e), but the same results. Regards

UPDATE

Error code from my first comment on @alexanderlukanin13 s answer:

Traceback (most recent call last): File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\pydevd.py", line 1570, in trace_dispatch Traceback (most recent call last): File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\pydevd.py", line 2278, in globals = debugger.run(setup['file'], None, None) File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\pydevd.py", line 1704, in run pydev_imports.execfile(file, globals, locals) # execute the script File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\runfiles.py", line 234, in main() File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\runfiles.py", line 78, in main return pydev_runfiles.main(configuration) # Note: still doesn't return a proper value. File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\pydev_runfiles.py", line 835, in main PydevTestRunner(configuration).run_tests() File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\pydev_runfiles.py", line 762, in run_tests file_and_modules_and_module_name = self.find_modules_from_files(files) File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\pydev_runfiles.py", line 517, in find_modules_from_files mod = self.__get_module_from_str(import_str, print_exception, pyfile) File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\pydev_runfiles.py", line 476, in __get_module_from_str buf_err = pydevd_io.StartRedirect(keep_original_redirection=True, std='stderr') File "C:\\Users\\Eddie\\Downloads\\eclipse\\plugins\\org.python.pydev_4.0.0.201504132356\\pysrc\\pydevd_io.py", line 72, in StartRedirect import sys MemoryError

Answer 1

The main problem in your script is that you don't save altered XML back to disk. You need to store reference to root element and then call ElementTree.write :

from xml.etree import ElementTree as Et

context = Et.iterparse('input.xml')
root = None
for event, elem in context:
    if elem.tag == 'DescriptorRecord':
        for e in list(elem.getchildren()):  # Don't use _children, it's a private field
            if e.tag in ['DateCreated', 'Year', 'Month', 'TreeNumber', 'HistoryNote', 'PreviousIndexing']:
                elem.remove(e)  # You need remove(), not clear()
    root = elem

with open('output.xml', 'wb') as file:
    Et.ElementTree(root).write(file, encoding='utf-8', xml_declaration=True)

Note: here I use an awkward (and probably unsafe) way to get a root element - I assume that it's always the last element in iterparse output. If anyone knows a better way, please tell.