简体   繁体   中英

Multiclass classifier neural network problem

I am implementing a Multiclass neural network that will recognize digits from the input images. The input images are five by five pixels which display five numbers. So, the neural network model contains 25 input nodes. 4 output nodes (as we have to classify five outputs) and 50 hidden nodes. I am using sigmoid function as an activation function for hidden nodes and softmax function as activation function for the output nodes.
My input argument X contains stacked two-dimensional image data ie 5*5*5 three dimensional matrices. So, I have converted into a (25,) vector using np.reshape(X[:,:,x],25) so that I can get the weighted sum easily (as the weights are 25*50 for input to hidden nodes and 50*5 for hidden to output nodes).

The problem I am facing is that when I calculate the Delta for input to hidden and hidden to output respectively, I have value error "ValueError: shapes (50,) and (5,5) not aligned: 50 (dim 0) != 5 (dim 0)" which I totally understand is because of the dimensions of both the arrays. But I am unable to figure out the solution so, that I can reshape them some way and take dot products. As I am learning python and this neural network stuff for the first time and writing my own code so, I don't have much expertise in playing with matrices. I need some help in 1) fixing this problem, 2) how can I improve this network (Future practices). 3) How can I make the network generic so that if I can add more layers etc I don't mess up with matrix multiplication with different dimensions? A prototype of code is below.

Code

# other stuff
def function(W1, W2, X, D):
 N = 5
 for x in range(N):
    # reshaping input in 25*1 vector
    l0 = np.reshape(X[:,:,x],25)
    ll = sigmoid(np.dot(l0,W1))
    l2 = softmax(np.dot(ll,W2))
    l2_error = D - l2
    l2_delta = l2_error
    l1_error = l2_delta.dot(W2.T)
    l1_delta = ll*(1-ll)*l1_error
    DW2 = alpha*ll.T.dot(l2_delta) #ValueError: shapes (50,) and (5,5) not aligned: 50 (dim 0) != 5 (dim 0)
    W2 = W2 + DW2
    DW1 = alpha*l0.T.dot(l1_delta) #ValueError: shapes (25,) and (5,50) not aligned: 25 (dim 0) != 5 (dim 0)
    W1 = W1 + DW1

# other stuff

X = np.zeros((5,5,5), dtype=int)
D = np.zeros((5,5), dtype=int)

X[:,:,0] = [[0 ,1, 1, 0, 0],
       [0 ,0, 1, 0, 0],
       [0 ,0, 1, 0, 0],
       [0 ,0, 1, 0, 0],
       [0 ,1, 1, 1, 0]]

X[:,:,1] = [[1 ,1, 1, 1, 0],
       [0 ,0, 0, 0, 1],
       [0 ,1, 1, 1, 0],
       [1 ,0, 0, 0, 0],
       [1 ,1, 1, 1, 1]]

X[:,:,2] = [[1 ,1, 1, 1, 0],
       [0 ,0, 0, 0, 1],
       [0 ,1, 1, 1, 0],
       [0 ,0, 0, 0, 1],
       [1 ,1, 1, 1, 0]]

X[:,:,3] = [[0 ,0, 0, 1, 0],
       [0 ,0, 1, 1, 0],
       [0 ,1, 0, 1, 0],
       [1 ,1, 1, 1, 1],
       [0 ,0, 0, 1, 0]]

X[:,:,4] = [[1 ,1, 1, 1, 1],
       [1 ,0, 0, 0, 0],
       [1, 1, 1, 1, 0],
       [0 ,0, 0, 0, 1],
       [1 ,1, 1, 1, 0]]

D = np.array([ [1,0,0,0,0],[0,1,0,0,0],[0,0,1,0,0],[0,0,0,1,0],[0,0,0,0,1] ]).T

W1 = 2*np.random.random((25,50)) - 1
W2 = 2*np.random.random((50,5)) - 1
W1, W2 = Multiclass(W1, W2, X, D)

these arrays (N,) are called first rank array. you will get unexpected values or error if you use them in calculation. to make them proper for use, np.reshape(N,1) fix it.but... it seems you have many holes in your understanding of neural networks, I suggest to do the following:

first you need some basic of linear algebra at least learn how to calculate the inner product of 2 matrix/vector on paper. khanacademy.org has a free linear algebra course which is surprisingly easy to follow.

second, take one machine learning course, Andrew Ng at coursera.org has invaluable materials. his famous machine learning course is awesome, but if you want neural networks take course 1 of his deep learning specialization. those are all free to audit.

last thing, don't reinvent the wheel. there are many free, popular, efficient and easy to use frameworks for machine learning. tensorflow, scikit-learn, pytorch, etc.

happy learning :)

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM