How I can parallelize a 'for' loop with an external variable?

Question

I am having difficulties parallelizing that part of code where new_text is of type unicode:

for old, new in self.replacements:
    line = pywikibot.replaceExcept(
        line, old, new, self.excsInside, self.site)
if new_text != entry.text:
    yield pywikibot.Page(self.site, entry.title)

The task looks to be easy with joblib or a process-pool , but there is new_text which is used outside of the loop. I have no idea of the equivalent of #pragama omp ordered or #pragma omp atomic , since there is no OpenMP wrapper for Python ...

How do I determine what the value of new_next is going to be in the if statement if it's run in parallel?

Answer 1

As it is inherently sequential, you can parallelize per line:

for line in new_text
    for old, new in self.replacements:
        line = pywikibot.replaceExcept(
            line, old, new, self.excsInside, self.site)

Ánd you parallelize the outer for loop (map with each replacement and then reduce by concatenation).