Exponential fit of the data (python)

Question

Hi I'm trying to fit my data with an either polynomial or exponential function which I failed in both. The code I'm using is as follows:

with open('argon.dat','r') as f:
    argon=f.readlines()

eng1 = np.array([float(argon[argon.index(i)].split('\n')[0].split('  ')[0])*1000 for i in argon])
II01 = np.array([1-math.exp(-float(argon[argon.index(i)].split('\n')[0].split('  ')[1])*(1.784e-3*6.35)) for i in argon])

with open('copper.dat','r') as f:
    copper=f.readlines()

eng2 = [float(copper[copper.index(i)].split('\n')[0].split('  ')[0])*1000 for i in copper]
II02 = [math.exp(-float(copper[copper.index(i)].split('\n')[0].split('  ')[1])*(8.128e-2*8.96)) for i in copper]

fig, ax1 = plt.subplots(figsize=(12,10))
ax2 = ax1.twinx()
ax1.set_yscale('log')
ax2.set_yscale('log')

arg = ax2.plot(eng1, II01, 'b--', label='Argon gas absorption at STP (6.35 cm)')
cop = ax1.plot(eng2, II02, 'r', label='Copper wall transp. (0.81 mm)')
plot = arg+cop

labs = [l.get_label() for l in plot]
ax1.legend(plot,labs,loc='lower right', fontsize=14)
ax1.set_ylim(1e-6,1)
ax2.set_ylim(1e-6,1)
ax1.set_xlim(0,160)
ax1.set_ylabel(r'$\displaystyle I/I_0$', fontsize=18)
ax2.set_ylabel(r'$\displaystyle 1-I/I_0$', fontsize=18)
ax1.set_xlabel('Photon Energy [keV]', fontsize=18)
plt.show()

Which gives me What I want to do is instead of drawing data like this fit them to an exponential curve and multiply these curves to end up with detector efficiency (I tried to multiply element by element but I don't have enough data points to have a smooth curve) I tried to use polyfit and also tried to define an exponential function to see f its working however I ended up with a line in both cases

#def func(x, a, c, d):
#    return a*np.exp(-c*x)+d
#    
#popt, pcov = curve_fit(func, eng1, II01)
#plt.plot(eng1, func(eng1, *popt), label="Fitted Curve")

and

model = np.polyfit(eng1, II01 ,5) 
y = np.poly1d(model)
#splineYs = np.exp(np.polyval(model,eng1)) # also tried this but didnt work
ax2.plot(eng1,y)

In case of need data is taken from http://www.nist.gov/pml/data/xraycoef/index.cfm Also similar work can be found in Fig.3 : http://scitation.aip.org/content/aapt/journal/ajp/83/8/10.1119/1.4923022

Rest is eddited after @Oliver's answer:

I did the multiplication by using existed data:

i = 0
eff1 = []
while i < len(arg):
    eff1.append(arg[i]*cop[i])
    i += 1 

What I ended up is (red: copper, dashed blue: argon, blue: multiplication) This is what I suppose to get but by using the functions for the curves this will be a smooth curve which I want to end up with (a comment has been made under @oliver's answer concerning what is wrong or misunderstood)

Answer 1

The reason curvefit is giving you a constant (a flat line), is because you're passing it a dataset that is uncorrelated using the model you have defined!

Let me recreate your setup first:

argon = np.genfromtxt('argon.dat')
copper = np.genfromtxt('copper.dat')

f1 = 1 - np.exp(-argon[:,1] * 1.784e-3 * 6.35)
f2 = np.exp(-copper[:,1] * 8.128e-2 * 8.96)

Now notice that f1 is based on the 2nd column of the data in the file argon.dat . It is NOT related to the first column, although nothing stops you from plotting a modified version of the 2nd column vs the first of course, and that is what you did when you plot:

import matplotlib.pyplot as plt
from scipy.optimize import curve_fit

plt.semilogy(copper[:,0]*1000, f2, 'r-')  # <- f2 was not based on the first column of that file, but on the 2nd. Nothing stops you from plotting those together though...
plt.semilogy(argon[:,0]*1000, f1, 'b--')
plt.ylim(1e-6,1)
plt.xlim(0, 160)

def model(x, a, b, offset):
    return a*np.exp(-b*x) + offset

Remark: in your model you had a parameter called b that was unused. That is always a bad idea to pass to a fitting algorithm. Get rid of it.

Now here's the trick: you made f1 based on the 2nd column, using an exponentional model. So you should pass curve_fit the 2nd column as the independent variable (which is labelled as xdata in the function's doc-string ) and then f1 as the dependent variable. Like this:

popt1, pcov = curve_fit(model, argon[:,1], f1)
popt2, pcov = curve_fit(model, cupper[:,1], f2)

And that will work perfectly well.

Now, when you want to plot a smooth version of the product of the 2 graphs, you should start from a common interval in the independent variable. For you, this is the photon energy. The 2nd column in both datafiles depends on that: there is a function (one for argon, another for copper) that relates the μ/ρ to the photon energy. So, if you have lots of datapoints for the energy, and you managed to get those functions, you will have many datapoints for μ/ρ . As those functions are unknown though, the best thing I can do is to simply interpolate. However, the data is logarithmic, so logarithmic interpolation is required, not the default linear.

So now, continue by getting lots of datapoints for the photon energy. In the dataset, the energypoints are exponentially increasing, so you can create a decent new set of points by using np.logspace :

indep_var = argon[:,0]*1000
energy = np.logspace(np.log10(indep_var.min()),
                     np.log10(indep_var.max()),
                     512)  # both argon and cupper have the same min and max listed in the "energy" column.

It works to our advantage that the energy in both datasets have the same minimum and maximum. Otherwise, you would have had to reduce the range of this logspace.

Next, we (logarithmically) interpolate the relation energy -> μ/ρ :

interpolated_mu_rho_argon = np.power(10, np.interp(np.log10(energy), np.log10(indep_var), np.log10(argon[:,1]))) # perform logarithmic interpolation
interpolated_mu_rho_copper = np.power(10, np.interp(np.log10(energy), np.log10(copper[:,0]*1000), np.log10(copper[:,1])))

Here's a visual representation of what has just been done:

f, ax = plt.subplots(1,2, sharex=True, sharey=True)
ax[0].semilogy(energy, interpolated_mu_rho_argon, 'gs-', lw=1)
ax[0].semilogy(indep_var, argon[:,1], 'bo--', lw=1, ms=10)
ax[1].semilogy(energy, interpolated_mu_rho_copper, 'gs-', lw=1)
ax[1].semilogy(copper[:,0]*1000, copper[:,1], 'bo--', lw=1, ms=10)
ax[0].set_title('argon')
ax[1].set_title('copper')
ax[0].set_xlabel('energy (keV)')
ax[0].set_ylabel(r'$\mu/\rho$ (cm²/g)')

The original dataset, marked with blue dots, has been finely interpolated.

Now, the last steps become easy. Because the parameters of your model that maps μ/ρ to some exponential variant (the functions that I have renamed as f1 and f2 ) have been found already, they can be used to make a smooth version of the data that was present, as well as the product of both these functions:

plt.figure()
plt.semilogy(energy, model(interpolated_mu_rho_argon, *popt1), 'b-', lw=1)
plt.semilogy(argon[:,0]*1000, f1, 'bo ')

plt.semilogy(copper[:,0]*1000, f2, 'ro ',)
plt.semilogy(energy, model(interpolated_mu_rho_copper, *popt2), 'r-', lw=1) # same remark here!

argon_copper_prod = model(interpolated_mu_rho_argon, *popt1)*model(interpolated_mu_rho_copper, *popt2)
plt.semilogy(energy, argon_copper_prod, 'g-')

plt.ylim(1e-6,1)
plt.xlim(0, 160)
plt.xlabel('energy (keV)')
plt.ylabel(r'$\mu/\rho$ (cm²/g)')

And there you go. To summarize:

1. generate a sufficient amount of datapoints of the independent variable to get smooth results
2. interpolate the relationship photon energy -> μ/ρ
3. map your function to the interpolated μ/ρ