How to add L1 norm to loss function in Keras or Tensorflow?

Question

I'm trying to reproduce the experiment results of a deep learning paper,

I notice that the loss function, category cross-entropy have a L1 regularization can be seen below,

So, how to compute the Model complexity and to add a right custom loss function in Keras/Tensorflow?

Answer 1

That is just a norm and might not necessarily be the 1-norm. Nevertheless, in tensorflow that is pretty straightforward if you have your parameters stored in a list (I will call it para_list):

l1_reg_term = sum([tf.reduce_sum(tf.abs(_var)) for _var in para_list])
reg_loss = loss + alpha * l1_reg_term

One way to get your parameters is to name them. Then you can just extract them from the global space with tf.get_global_variables() or tf.get_trainable_variables().