将高斯拟合到一维蒙版数据数组

Question

I have a masked 1D data array which contains nan values in it which I have masked and now print as --. I wish to fit this array to a Gaussian and create a histogram using the mean and standard deviation of the fit. I've tried spicy.stats.fit but that didn't work (mean and std just returned 'nan'). I then sought after spicy.mstats but it doesn't look like it has a fit function.

Is there a module that can fit a Gaussian to a masked array and output a mean and std?

EDIT: Here is my code thusfar

def createRmsMatrix( self ):

    '''
    Creates an array of RMS values for each profile in one file.
    '''

    # Initialize RMS table of zeroes
    rmsMatrix = np.zeros( ( self.nSub, self.nChan ), dtype = float )

    # Loop over the time and frequency indices
    for time in np.arange( self.nSub ):
        for frequency in np.arange( self.nChan ):

            # Create a mask along the bin space
            mask = utils.binMask( self.data[time][frequency], 0.55 )

            #print(mask)

            rmsMatrix[time][frequency] = mu.rootMeanSquare( self.data[time][frequency][mask == 0] )

    # Mask the nan values in the array
    rmsMatrix = np.ma.array( rmsMatrix, mask = np.isnan( rmsMatrix ) )

    print( "Root Mean Square matrix created..." )

    return rmsMatrix

And the part of my main function that calls this is:

    # Return the array of RMS values for each profile
    self.rmsArray = self.createRmsMatrix()

    # Reshape RMS array to be linear and store in a new RMS array
    self.linearRmsArray = np.reshape( self.rmsArray, ( self.nChan * self.nSub ) )

    # Best fit of data using a Gaussian fit
    mu, sigma = norm.fit( self.linearRmsArray )

    # Creates the histogram
    n, bins, patches = self.histogramPlot( self.linearRmsArray, mu, sigma, 'Root Mean Squared', 'Frequency Density', True )

The histogramPlot is just a convenient matplotlib organizer for me which I will also post:

def histogramPlot( self, data, mean, stdDev, xAxis='x-axis', yAxis='y-axis', showPlot = False ):

    '''
    Plots and returns a histogram of some linear data using matplotlib
    and fits a Gaussian centered around the mean with a spread of stdDev.
    Use this function to set the x and y axis names.
    Can also toggle showing of the histogram in this function.
    '''

    # Plot the histogram
    n, bins, patches = plt.hist( self.linearRmsArray, bins=self.nChan, normed=True )

    # Add a 'best fit' normal distribution line
    xPlot = np.linspace( ( mean - (4*stdDev) ), ( mean + (4*stdDev) ), 1000 )
    yPlot = mlab.normpdf( xPlot, mean, stdDev )
    l = plt.plot(xPlot, yPlot, 'r--', linewidth=2)

    # Format axes
    plt.ylabel( yAxis )
    plt.xlabel( xAxis )
    #plt.title(r'$\mathrm{Histogram\ of\ data:}\ \mu=%.3f,\ \sigma=%.3f$' %(mu, sigma))
    plt.title(r'$\mu=%.3f,\ \sigma=%.3f$' %(mean, stdDev))
    plt.grid(True)

    if showPlot == True:
        plt.show()

    return n, bins, patches

Answer 1

You were attempting to use scipy.norm.fit to fit a normal distribution to your data, which implies that your input is collection of values that is supposed to be a random sample from a normal distribution. In that case, the maximum likelihood estimates of the mean and std. dev. are simply the sample mean and sample standard deviation of the data. For data that contains nan , you could remove the nan s before calling scipy.norm.fit() , or you can compute these directly with numpy.nanmean and numpy.nanstd :

est_mean = np.nanmean(data)
est_stddev = np.nanstd(data)

For example,

In [18]: import numpy as np

In [19]: from scipy.stats import norm

In [20]: x = np.array([1, 4.5, np.nan, 3.3, 10.0, 4.1, 8.5, 17.1, np.nan])

In [21]: np.nanmean(x), np.nanstd(x)
Out[21]: (6.9285714285714288, 5.0366412520687653)

In [22]: norm.fit(x[np.isfinite(x)])
Out[22]: (6.9285714285714288, 5.0366412520687653)

Note that x[np.isfinite(x)] is the array of values in x that are not nan or inf .

If you have a masked array, you can use the mean and std methods:

In [36]: mx = np.ma.masked_array(x, np.isnan(x))

In [37]: mx
Out[37]: 
masked_array(data = [1.0 4.5 -- 3.3 10.0 4.1 8.5 17.1 --],
             mask = [False False  True False False False False False  True],
       fill_value = 1e+20)

In [38]: mx.mean(), mx.std()
Out[38]: (6.9285714285714288, 5.0366412520687653)