如何使神经网络功能更快？

Question

I have this function for a neural network and it's the function to calculate the next layer from a list of inputs and a list of weights. Is there any way to make this faster or more efficient? the arguments inp is the input, weights are the weights, layerlength is the length of the next layer and rounds is just the length to round the output to.

def output(inp,weights,layerlength,rounds):
    layer=[]
    count=0
    lappend=layer.append
    for a in range(layerlength):
        total=0
        for b in range(len(inp)):  
            total+=inp[b]*weights[count]
            count+=1
        lappend(round(total,rounds))
    return layer

Answer 1

In general, try not to use for loop constructs in Python. They are extremely slow. Use Matrix operations programmed with numpy instead, then the loops will run under the hood in C++ instead (50 to 100 times faster). You can easily reformulate your above piece of code without any Python for loops by defining your layer and inp vectors and your weights matrix all as numpy.array() and then perform matrix multiplication on them.

EDIT: I hope I am not helping you cheat on your homework here ;)

import numpy as np
# 10 dimensional input
inpt = np.arange(10)
# 20 neurons in the first (fully connected) layer
weights = np.random.rand(10, 20)
# mat_mul: to calculate the input to the non-linearity of the first layer
# you need to multiply each input dimension with all the weights assigned to a specific neuron of the first layer
# and then sum them up, and this for all the neurons in that layer
# you can do all of that in this single Matrix multiplication
layer = np.matmul(inpt, weights)
print(inpt.shape)
print()
print(weights.shape)
print()
print(layer.shape)

Answer 2

So I'm assuming you're computing the activations of one layer.

Make sure you use linear algebra libraries like Numpy (or Tensorflow, PyTorch, etc). These will make sure your computations run much more efficient on the CPU (or GPU). Typically using for loops give a lot of computational overhead.

For example, in numpy you can write your feedforward pass for one layer as:

output = inp.dot(weights)

inp is here your n by m input matrix, weights is your m by k weight matrix. output will then be a n by k matrix of your forward step activations.