Simple neural network - how to store weights?

Question

I recently started learning Python and am trying to implement my first neural network. My goal is to write a function that generates a neural net with a variable amount of layers and nodes. All necessary information for that is stored in layerStructure (eg: First layer has four nodes, third layer has three nodes).

import numpy as np

#Vector of input layer
input = np.array([1,2,3,4])

#Amount of nodes in each layer
layerStructure = np.array([len(input),2,3])

#Generating empty weight matrix container
weightMatrix_arr = np.array([])

#Initialsing random weights matrices
for ii in range(len(layerStructure[0:-1])):
    randmatrix = np.random.rand(layerStructure[ii+1],layerStructure[ii])
    print(randmatrix)

The code above generates the following output:

[[0.6067148  0.66445212 0.54061231 0.19334004]
 [0.22385007 0.8391435  0.73625366 0.86343394]]
[[0.61794333 0.9114799 ]
 [0.10626486 0.95307027]
 [0.50567023 0.57246852]]

My first attempt was to store each random weight matrix in a container array called weightMatrix_arr . However, since the shape of individual matrices varies, I cannot use np.append() to store them all in the matrix container. How can I save these matrices in order to access them during the backpropagation?

Answer 1

You can use a list instead of an np.array :

#Generating empty weight LIST container
weightMatrixes = []

#Initialsing random weights matrices
for ii in range(len(layerStructure[0:-1])):
    randmatrix = np.random.rand(layerStructure[ii+1],layerStructure[ii])
    weightMatrixes.append(randmatrix)
    print(randmatrix)

Otherwise you can set the weightMatrix_arr dtype to object : :

#Generating empty weight LIST container
weightMatrixes = np.array([], dtype=object)

#Initialsing random weights matrices
for ii in range(len(layerStructure[0:-1])):
   randmatrix = np.random.rand(layerStructure[ii+1],layerStructure[ii])
   weightMatrixes = np.append(weightMatrixes, randmatrix)

Note both ways you can't access the inner layer indexes without accessing the layer matrix:

weightMatrixes[layer, 0, 3] # ERROR
weightMatrixes[layer][0, 3] # OK

Answer 2

If memory consumption is not a problem, you can shape all layers as a longest one, and just ignore extra cells according to a layerStructure value.