Multiprocessing and matlibplot, non-blocking plots

Question

I know there are quite a few other questions on this topic and i have read/experimented with the solutions but none of them quite do what i want or dont work ...

I would like to create a plot using matplotlib with the plot launched in a new process such that it is open and responsive while the main thread is running, additionally when the main thread exits not to kill the child processes. I would like something portable if possible hence i have avoided fork so far as my impression is that it isn't (correct?). I have tried the show(block=False) in matplotlib. If there isn't anything else im happy to accept that but preferably the main thread exits and the plots are still sitting there.

This code is modified from Python close children when closing main process

import logging, signal, sys, time
import multiprocessing as mp 
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid.axislines import SubplotZero

class AddProcessNameFilter(logging.Filter):
    """Add missing on Python 2.4 `record.processName` attribute."""
    def filter(self, r):
        r.processName = getattr(r, 'processName', mp.current_process().name)
        return logging.Filter.filter(self, r)

def print_dot(plt):
    signal.signal(signal.SIGTERM, signal.SIG_IGN)
    while True:
        mp.get_logger().info(".")
        plt.show(block=True)

def main():
    logger = mp.log_to_stderr()
    logger.setLevel(logging.INFO)
    logger.addFilter(AddProcessNameFilter()) # fix logging records

    fig = plt.figure()
    ax = fig.add_subplot(111)
    ax.plot([0, 1, 2, 3], [1, 1, 3, 2],'ro')

    # create daemonic child processes
    p = mp.Process(target=print_dot, args=(plt,))
    p.daemon = True
    p.start()    

if __name__=="__main__":
    mp.freeze_support()
    main()

The plot isn't launched additionally the main thread doesn't exit it just waits on join. When interrupted I get the following error as well ...

X Error of failed request:  BadIDChoice (invalid resource ID chosen for this connection)
  Major opcode of failed request:  53 (X_CreatePixmap)
  Resource id in failed request:  0x4400017
  Serial number of failed request:  344
  Current serial number in output stream:  352

Not really sure what to make of that. Any ideas? Thanks in advance :D

PS: Ubuntu 12.04 ... but i don't know if thats particularly relevant in this case ...

Answer 1

I use the following code. It's similar to yours, but joins old processes as the function is supposed to be called many times. You can close the main process and the plot won't be closed.

There's no guarantee that the plot will be shown just after the p.start is called. But this would be very unusual and happened to me just few times. And in these times the window is shown either if plot is called again or if I call p.is_alive().

    import multiprocessing as mp
    import sys
    def foo(*arg):
        import matplotlib.pyplot as plt
        plt.ioff()    
        plt.plot(*arg)
        plt.show()

    if __name__ == '__main__':
        mp.freeze_support()
        arg=(range(10),)
        global pl

        #create the process list, if it does not exists
        try:
            pl
        except NameError:
            pl = []    

        #Join old ones to keep zombie process list small
        for i in range(len(pl)):
            if not (pl[i] is None):
                if not pl[i].is_alive():
                    pl.pop(i).join()


        #create plot process
        p = mp.Process(target=foo, args=arg)
        p.start()
        pl.append(p)

Answer 2

I recommend a solution from https://stackoverflow.com/a/62354049/13995577 . Works well for me. It is a bit of a hack though, but relatively small.

import multiprocessing as mp
def my_plot_routine(q):
    ...
    q.put('Done, see plot window')
    plt.show()
    return

class EverLastingProcess(mp.Process):
    def join(self, *args, **kwargs):
        pass

    def __del__(self):
        pass

if __name__ == '__main__':
    # create plot process
    mp.set_start_method('spawn')
    q = mp.Queue()
    p = EverLastingProcess(target=my_plot_routine, args=(q,))
    p.start()
    print(q.get())