Scipy lognorm fitting to histogram

Question

I'm fitting a lognormal pdf to some binned data, but my curve doesn't quite match the data, see image below. My code is:

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import lognorm

data = genfromtxt('data.txt')
data = np.sort(data)

# plot histogram in log space

ax.hist(data, bins=np.logspace(0,5,200),normed=1)
ax.set_xscale("log")

shape,loc,scale = lognorm.fit(data)

print shape, loc, scale

pdf = sp.stats.lognorm.pdf(data, shape, loc, scale)

ax.plot(data,pdf)

plt.show()

This is what it looks like:

Do I need to somehow provide the fit with sensible guesses for shape, loc and scale?

Thanks!

Answer 1

The data you are trying to fit does not look like a lognormal distribution. The lognormal distribution, when plotted on a logarithmic x scale should look like a normal distribution. This is not the case in the plot you show. When the distribution does not fit the data well you get weird parameters.

You will need to find out how your data is really distributed (which, strictly speaking, is off-topic at SO) before attempting to fit something.

This is what we get when using data randomly drawn from a lognormal distribution:

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import lognorm

np.random.seed(42)

data = lognorm.rvs(s=0.5, loc=1, scale=1000, size=1000)

# plot histogram in log space
ax = plt.subplot(111)
ax.hist(data, bins=np.logspace(0,5,200), density=True)
ax.set_xscale("log")

shape,loc,scale = lognorm.fit(data)

x = np.logspace(0, 5, 200)
pdf = lognorm.pdf(x, shape, loc, scale)

ax.plot(x, pdf, 'r')

plt.show()