Add l1 or l2 regularization to crossentropy() function

Question

The following command is the cross entropy function in 5_convolutional_net.py . I want to add L1 or L2 regularization to this cost function. I do not know why the TypeError: bad operand type for abs(): 'list' comes up ?

def RMSprop(cost, params, lr=0.001, rho=0.9, epsilon=1e-6):
grads = T.grad(cost=cost, wrt=params)
updates = []
for p, g in zip(params, grads):
    acc = theano.shared(p.get_value() * 0.)
    acc_new = rho * acc + (1 - rho) * g ** 2
    gradient_scaling = T.sqrt(acc_new + epsilon)
    g = g / gradient_scaling
    updates.append((acc, acc_new))
    updates.append((p, p - lr * g))
return updates

cost = T.mean(T.nnet.categorical_crossentropy(noise_py_x, Y)) params = [w, w2, w3, w4, w_o] updates = RMSprop(cost, params, lr=0.001)

Once used cost+=T.sum(abs(params)) it gives me TypeError: bad operand type for abs(): 'list'

Answer 1

您不能将abs（）应用于列表，而是应用于列表的每个元素：

cost+=T.sum([abs(p) for p in params])