How do I update a matplotlib figure while fitting a function?

Question

I have written a python script which opens UV-Vis spectra and attempts to fit them with a large sum of functions. However, I would like the fitting steps to be shown in a plot as the minimum residual is being found. Stackoverflow actually has some examples that touch on this idea (http://stackoverflow.com/questions/4098131/matplotlib-update-a-plot), but for some reason this approach isn't working for me very well. What I mean by "isn't working very well" is that the plot window does not respond to the updates that are occurring in the script. I have attempted to cut down my code to something that is more understandable, still compiles, but is also closer to the code I have than the example and it is shown below.

To re-phrase my question: Is there a better way of doing this type of screen refreshing through a fitting process so that the window does not become "(Not Responding)"?

Here is my simplified code:

# import modules that I'm using
import matplotlib
matplotlib.use('GTKAgg')
import Tkinter
from Tkinter import *
import numpy as np
import scipy as sc
import matplotlib.pyplot as pltlib
# lmfit is imported becuase parameters are allowed to depend on each other along with bounds, etc.
from lmfit import minimize, Parameters, Minimizer

#If button is pressed on the window, open a file and get the data
def open_File():
    # file is opened here and some data is taken
    # I've just set some arrays here so it will compile alone
    x=[]
    y=[]
    for num in range(0,1000):x.append(num*.001+1)
    # just some random function is given here, the real data is a UV-Vis spectrum
    for num2 in range(0,1000):y.append(sc.math.sin(num2*.06)+sc.math.e**(num2*.001))
    X = np.array(x)
    Y = np.array(y)

    # plot the initial data in one figure
    pltlib.ion()
    pltlib.interactive(True)
    pltlib.figure(1)
    pltlib.plot(X,Y, "r-")
    pltlib.show()

    #deconvolute this initial data into deveral lorentzian profiles
    deconvolute(X,Y)

#lorentz line for constructing the more complex function
def lorentz(x, amp, center, width):
    return amp*1/sc.math.pi*(width/((x-center)**2+width**2))

def deconvolute(X,Y):
    #make 2nd figure for the refreshing screen
    deconvFig = pltlib.figure(2)
    ax = deconvFig.add_subplot(111)
    line1,line2=ax.plot(X,Y,'r-',X,Y,'r-')

    # setup parameters for several (here is 30, I potentially hae many more in the real program)
    params = Parameters()
    for p in range(0,30):
        params.add('amp' + str(p), value=1)
        params.add('center' + str(p), value=1)
        params.add('width' + str(p), value=1)

    #get residual function for minimizing
    def residual(params, X, data=None):
        model = 0
        # get values for each lorentz and sum them up
        for p in range(0,30):
            amp = params['amp' + str(p)].value
            center = params['center' + str(p)].value
            width = params['width' + str(p)].value
            tmpLorentz = lorentz(X, amp, center, width)
            model = model + tmpLorentz

        # This is where the main problem is.
        # This 2nd figure stops responding after a very small (1?) number of iterations
        ########################################
        # I want redraw the figure at every step through the fitting process
        line2.set_ydata(model)
        deconvFig.canvas.draw()
        print 'screen should be refreshed'
        ########################################

        return (data - model)

    #fit the function to the data
    result = minimize(residual, params, args=(X, Y))
    print 'done fitting the program'

#create a window with a button
MainWindow = Tk()
Button(text='Open a File', command=open_File).pack(side=BOTTOM)
MainWindow.mainloop()

Answer 1

interesting, I tried running a simple test.

import time
from matplotlib import pyplot as pltlib
deconvFig = pltlib.figure(2)
ax = deconvFig.add_subplot(111)
X, Y = range(10), range(10)
line1,line2 = ax.plot(X,Y,'r-',X,Y,'r-')
for x in xrange(2, 6, 1):
    line2.set_ydata(range(0, 10*x, x))
    deconvFig.canvas.draw()
    time.sleep(2)

>>> import matplotlib
>>> matplotlib.__version__
'1.1.0'

and well it worked as expected.
maybe because your generating a second figure.

import time
from matplotlib import pyplot as pltlib

pltlib.ion()
pltlib.interactive(True)
pltlib.figure(1)
pltlib.plot(range(10),range(10), "r-")
pltlib.show()

deconvFig = pltlib.figure(2)
ax = deconvFig.add_subplot(111)
X, Y = range(10), range(10)
line1,line2 = ax.plot(X,Y,'r-',X,Y,'r-')
for x in xrange(2, 6, 1):
    line2.set_ydata(range(0, 10*x, x))
    deconvFig.canvas.draw()
    time.sleep(2)

nope still worked fine.
It could be my setup.

Though its also possible that its minimizing at very slow rate, so when you plot the update you can't tell the difference, you can calculate the RMSE to see how big the difference is

print numpy.sqrt(numpy.sum((data - model)**2)/model.shape[0])/numpy.mean(data) * 100  

Also I usually use scipy's minimization function http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html being that it can minimize most functions, though it works by randomly mutating the input so I don't know how fast it can be, but it can be applied in many many situations.

I hope this helps.