具有高级计算功能的Keras自定义图层

Question

I want to write some custom Keras Layers and do some advanced calculations in the layer, for example with Numpy, Scikit, OpenCV...

I know there are some math functions in keras.backend that can operate on tensors, but i need some more advanced functions.

However, i have no clue how to implement this correctly, i get the error message:
You must feed a value for placeholder tensor 'input_1' with dtype float and shape [...]

Here is my custom layer:

class MyCustomLayer(Layer):
    def __init__(self, **kwargs):
        super(MyCustomLayer, self).__init__(**kwargs)

    def call(self, inputs):
        """
        How to implement this correctly in Keras?
        """
        nparray = K.eval(inputs)  # <-- does not work
        # do some calculations here with nparray
        # for example with Numpy, Scipy, Scikit, OpenCV...
        result = K.variable(nparray, dtype='float32')
        return result

    def compute_output_shape(self, input_shape):
        output_shape = tuple([input_shape[0], 256, input_shape[3]])
        return output_shape  # (batch, 256, channels)

The error appears here in this dummy model:

inputs = Input(shape=(96, 96, 3))
x = MyCustomLayer()(inputs)
x = Flatten()(x)
x = Activation("relu")(x)
x = Dense(1)(x)    
predictions = Activation("sigmoid")(x)
model = Model(inputs=inputs, outputs=predictions)

Thanks for all hints...

Answer 1

TD;LR You should not mix Numpy inside Keras layers. Keras uses Tensorflow underneath because it has to track all the computations to be able to compute the gradients in the backward phase.

If you dig in Tensorflow, you will see that it almost covers all the Numpy functionality (or even extends it) and if I remember correctly, Tensorflow functionality can be accessed through the Keras backend (K).

What are the advance calculations/functions you need?

Answer 2

i think that this kinda process should apply before the model because the process does not contain variables so it cant be optimized.

K.eval(inputs) does not work beacuse you are trying to evaluate a placeholder not variable placeholders has not values for evaluate. if you want get values you should feed it or you can make a list from tensors one by one with tf.unstack()

nparray = tf.unstack(tf.unstack(tf.unstack(inputs,96,0),96,0),3,0)

your call function is wrong because returns a variable you should return a constant:

result = K.constant(nparray, dtype='float32')
return result