在Tensorflow中训练网络时忽略特定的输入损耗

Question

I have a binary classification problem in which only the beginning 3 time instances of positive events are annotated in the data set, and I have apriori knowledge that they could last up to 15 time instances. To cope with this issue, I have decided to disregard loss values from the 15-3 = 12 time instances following an annotation when training the network.

I am training an LSTM network using Tensorflow in Python, and my training batches have sequence_len=240 , with the positive events likely to occur at any time on the sequence in any iteration.

Basically, my cost metric is (using AdamOptimizer )

loss = tf.nn.softmax_cross_entropy_with_logits(logits, self._targets)
cost = tf.reduce_mean(loss)

I am thinking I simply need to remove the undesired elements from loss before passing them to tf.reduce_mean() . I developed an algorithm to do a desired mask under the assumption of targets being a numpy array:

v = targets[:, 0]
w = np.where(np.multiply(v[:-1] == 1, v[1:] < 1))[0]
m = np.ones(v.shape, dtype=bool)
    for i in w:
    i1 = i + 1
    i2 = np.min((i + 13, len(v)))
    ind = np.arange(i1, i2, dtype=np.int)
    m[ind] = False
return m

This algorithm works, but not in Tensorflow scope! Obviously because the input is then a tensor and not a numpy array.

Hence, my question: How do I migrate this small algorithm/mask to Tensorflow?

Answer 1

Quickly after posting this question I came up with the obvious answer myself. I will include a placeholder for "error weights" in the network:

self._mask = tf.placeholder(tf.float32, [None])

Then in the cost function I will perform a weighted loss on these.

loss = tf.nn.softmax_cross_entropy_with_logits(logits, self._targets)
cost = tf.reduce_sum(tf.mul(loss, self._mask) / tf.reduce_sum(self._mask)

And then generate the mask alongside my input data. This is also better because I can make up for positive events occuring just prior to a new sequence, which the previous solution would not be able to.