多处理和多线程的简单示例

Question

I have the following code:

class SplunkUKAnalyser(object):

    def __init__
    def method1
    def method2
    def method2
    ...

class SplunkDEAnalyser(SplunkUKAnalyser):

    def __init__    (Over-ridden)
    def method1     (Over-ridden)
    def method2
    def method2
    ...

perform_uk_analysis():
    my_uk_analyser = SplunkUKAnalyser() 

perform_de_analysis():
    my_de_analyser = SplunkDEAnalyser()

It all works well if I just execute the below:

perform_uk_analysis()
perform_de_analysis()

How can I make it so that the two last statements are executed concurrently. (using mutliprocessing and/or multi-threading)?

From my test it seems that the second statement executes even though the first statement has not finished completely but I would like to incorporate true concurrency.

Any other additional advice is much appreciated.

Many thanks in advance.

Answer 1

Because of GIL (Global Interpreter Lock) you can not achieve 'true concurrency' with threading .

However, using multiprocessing to concurrently run multiple tasks is easy:

import multiprocessing

process1 = multiprocessing.Process(target=perform_uk_analysis)
process2 = multiprocessing.Process(target=perform_de_analysis)
# you can optionally daemoize the process
process2.daemon = True
# run the tasks concurrently
process1.start()
process2.start()
# you can optionally wait for a process to finish
process2.join()

For tasks that run the same function with different arguments, consider using multiprocessing.Pool , an even more convenient solution.