如何在Tensorflow的神经网络层中实现不同的激活功能？

Question

The following line creates a layer of size three with the sigmoid activation function on each neuron:

out = layers.dense(inputs=inp, units=3, activation=sigmoid)

What I would like to do is something like this:

out = layers.dense(inputs=inp, units=3, activation=[sigmoid sigmoid relu])

In essence, the first two neurons contain the sigmoid activation function and the third neuron contains the relu activation function.

My question is: How do I implement this?

I would appreciate it if someone could answer this question.

Answer 1

The easiest and cleanest way is to just create 2 outputs layers:

sigmoid_out = layers.dense(inputs=inp, units=2, activation=tf.nn.sigmoid)
relu_out = layers.dense(inputs=inp, units=1, activation=tf.nn.relu)

You can then concat both layers if you want :

out = tf.concat([sigmoid_out, relu_out], axis=1)