How to apply sigmoid function for each outputs in Keras?

Question

This is part of my codes.

model = Sequential()
model.add(Dense(3, input_shape=(4,), activation='softmax'))
model.compile(Adam(lr=0.1),
          loss='categorical_crossentropy',
          metrics=['accuracy'])

with this code, it will apply softmax to all the outputs at once. So the output indicates probability among all. However, I am working on non-exclusive classifire, which means I want the outputs to have independent probability. Sorry my English is bad... But what I want to do is to apply sigmoid function to each outputs so that they will have independent probabilities.

Answer 1

There is no need to create 3 separate outputs like suggested by the accepted answer.

The same result can be achieved with just one line:

model.add(Dense(3, input_shape=(4,), activation='sigmoid'))

---

You can just use 'sigmoid' activation for the last layer:

from tensorflow.keras.layers import GRU
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation
import numpy as np

from tensorflow.keras.optimizers import Adam

model = Sequential()
model.add(Dense(3, input_shape=(4,), activation='sigmoid'))
model.compile(Adam(lr=0.1),
          loss='categorical_crossentropy',
          metrics=['accuracy'])

pred = model.predict(np.random.rand(5, 4))
print(pred)

Output of independent probabilities:

[[0.58463055 0.53531045 0.51800555]
 [0.56402034 0.51676977 0.506389  ]
 [0.665879   0.58982867 0.5555959 ]
 [0.66690147 0.57951677 0.5439698 ]
 [0.56204814 0.54893976 0.5488999 ]]

---

As you can see the classes probabilities are independent from each other. The sigmoid is applied to every class separately.

Answer 2

You can try using Functional API to create a model with n outputs where each output is activated with sigmoid .

You can do it like this

in = Input(shape=(4, ))

dense_1 = Dense(units=4, activation='relu')(in)

out_1 = Dense(units=1, activation='sigmoid')(dense_1)
out_2 = Dense(units=1, activation='sigmoid')(dense_1)
out_3 = Dense(units=1, activation='sigmoid')(dense_1)

model = Model(inputs=[in], outputs=[out_1, out_2, out_3])