Windows上的Python 2.7多重处理

Question

I am new to multiprocessing, but I need to parse a large number of xml files, the parsing is part of a larger application. Parsing can happen in the "background" so it doesn't affect the main application. To that end I have the following (exerpt)

from lxml import etree
from StringIO import StringIO
import multiprocessing as mp


def parseBookXML(xmlFile):

    f = open(xmlFile)
    xml = f.read()
    f.close()

    tree = etree.parse(StringIO(xml))

    #now write it out to file
    with open("out_filename", "w") as fout:
    fout.write(ET.tostring(tree))

def do_process_file():
     process = mp.Process(target=parseBookXML, args=(c:\my_xml_file.xml))
     process.start()

Essentially I want to call the parseBookXML function, pass it a file as a parameter (or even a list of xml files) and let the parsing happen it its own process and complete.

When doing this normally (ie without process.start()) it works fine.

def do_process_file():
     self.parseBookXML(c:\my_xml_file.xml)

The above works fine, the xml_file is parsed no errors.

But making use of the do_process_file function gives me errors, along the lines of

AttributeError no module etree

I am on Windows using python 2.7, not using the REPL or interactive interpreter. I have read that windows differs from linux,

I have read a little about mulitprocessing not pickling in Windows (i'm not sure what that means)

what I really need to know is how to make this kind of function work.

Answer 1

from lxml import etree seems like your python does not have module lxml in it. Some module are different based on OS. Please check whether module lxml is available for windows or not.