Keras 中的正则化层不起作用

Question

I try to write a simple custom layer for regularization in Keras as follow:

from tensorflow.keras.regularizers import Regularizer
import tensorflow as tf

class MyRegularization(Regularizer):
      def __init__(self):
          self.alpha = 100000000

      def __call__(self, w):
          return self.alpha * tf.reduce_sum(w ** 2)

as you see the coefficent of regularization is so great. Then, I add this regularization to each layer of a simple network:

model = Sequential([
      Dense(1000, activation='relu', input_shape=[10,], kernel_regularizer=MyRegularization()),
      Dense(100, activation='relu', kernel_regularizer=MyRegularization()),
      Dense(10, activation='relu', kernel_regularizer=MyRegularization()),
      Dense(2, activation='softmax', kernel_regularizer=MyRegularization()),
])

I expect that because of a great alpha the learning is not converge, but it is look like the value of alpha does not affect the training procdeure. Why?

I took the alpha = 1e-4 but the there is not different when alpha=1e+10 . :-\

Answer 1

Are you calling the the class instance

MyRegularization()

or are you calling an object instance of the class with the same class name?

MyRegularization = MyRegularization()
...
MyRegularization()

because only the latter one will define the function body ( __call__ ) that's carried out when an instance is used as a function. That would explain why the regularization value are not returning, thus looking like the values of alpha are not affecting the training.