CNN predicting negative values

Question

I'm not sure, what I'm looking for and that's why I might miss the answer on the inte.net. I have fully convolutional neural.net U-Net. The paddings are always "same", the activation function is "relu". I'm feeding images to it with pixels between 0 and 1. The loss function is Binary Cross entropy since I have only 1 class. Optimizer is ADAM, metric is "accuracy" by default. When I left accuracy there, it works fine. However, when I change the metric to IOU, the code crashes with "negative values in prediction"

tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument:  assertion failed: [`predictions` contains negative values] [Condition x >= 0 did not hold element-wise:] [x (confusion_matrix/Cast:0) = ] [0 0 0...]
     [[{{node confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/else/_10/confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert}}]]
     [[confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/branch_executed/_17/_87]]
  (1) Invalid argument:  assertion failed: [`predictions` contains negative values] [Condition x >= 0 did not hold element-wise:] [x (confusion_matrix/Cast:0) = ] [0 0 0...]
     [[{{node confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/else/_10/confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert}}]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_4589]

So, I used "accuracy" for training and checked the prediction image and truly there are negative values predictid. Why?

Answer 1

I found out that someone tried to change the number of classes in the IOU metric from Keras and it helped. See: https://github.com/tensorflow/models/issues/8138

I tried that as well and changed the number of classes from 1 to 2. However, what you need to do is to change it larger number. 10+ worked for me.

Still, I don't know why.

Answer 2

Found out the problem. Pretty obvious. For some reason, I thought the default activation is "sigmoid", hence the last convolutional layer does not have specified "activation" unit. And even if the all activation units are relu, due to convolution, there might me negative values in the last layer. :facepalm: