可以将scipy.optimize.minimize用于Python中的神经网络

Question

I created a Neural Network with one hidden layer. Can I use the minimize function? What is the approach to pass two Theta matrix to mimimize. I am trying by using flatten on both the Theta and then concatenate them using

np.concatenate.
thetaFlat = np.concatenate((initial_Theta1.flatten(),initial_Theta2.flatten()),axis=0)

In my Cost and gradient function I am retrieving the thetas based on index and then reshaping them.

theta1=(thetaFlat[0:(hiddenLayerSize*(inputLayerSize+1))]).reshape(hiddenLayerSize,(inputLayerSize+1))

theta2=(thetaFlat[(hiddenLayerSize*(inputLayerSize+1)):len(thetaFlat)]).reshape(num_labels,(hiddenLayerSize+1))

Is this approach correct? This approach is however used in Octave, but the flattening there is columnwise and in Python its rowise.

The performance on training set is around 80% in Python. While in Octave its 96%.So I am a bit skeptical on the approach. Please also suggest any better option if available for Neural Network on Python.Or any other optimizaton function

regards Souvik Saha Bhowmik

Answer 1

This is a valid, and pretty much only approach. Remember to double check that your flattening and unflattening operations are each others inverses though (just define them as two functions, flat and unflat and check whether x == unflat(flat(x)) for many random x's). There are many possible reasons for differences in the final model, but flattening is not one of them.