crash in ctc loss function

Question

I am getting an the following InvalidArgumentError using ctc-loss function in Tensorflow 1.2.0-rc0 (python 2.7):

InvalidArgumentError (see above for traceback): label SparseTensor is not valid: indices[7] = [0,7] is out of bounds: need 0 <= index < [1,7]
         [[Node: loss/CTCLoss = CTCLoss[ctc_merge_repeated=true, ignore_longer_outputs_than_inputs=false, preprocess_collapse_repeated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](output_fc/BiasAdd/_91, _arg_labels/indices_0_1, _arg_labels/values_0_3, seq_len/Cast/_93)]]
         [[Node: loss/CTCLoss/_103 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_103_loss/CTCLoss", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

I do not understand the:

label SparseTensor is not valid: indices[7] = [0,7] is out of bounds: need 0 <= index < [1,7]

Since the SparseTensor (target sequence) that is crashing is (I set a batch size of 1):

(array([[ 0,  0],
       [ 0,  1],
       [ 0,  2],
       [ 0,  3],
       [ 0,  4],
       [ 0,  5],
       [ 0,  6],
       [ 0,  7],
       [ 0,  8],
       [ 0,  9],
       [ 0, 10],
       [ 0, 11],
       [ 0, 12],
       [ 0, 13],
       [ 0, 14],
       [ 0, 15],
       [ 0, 16],
       [ 0, 17],
       [ 0, 18],
       [ 0, 19],
       [ 0, 20],
       [ 0, 21],
       [ 0, 22]], dtype=int32), array([41,  2,  7,  0, 13, 19,  4, 11, 11,  4, 40, 19,  8,  1,  1, 18, 40,
       24,  4,  0,  7, 40, 41], dtype=int32), array([ 1, 23], dtype=int32))

In previous iterations it has processed longer sequences with similar values such as:

(array([[  0,   0],
       [  0,   1],
       [  0,   2],
       [  0,   3],
       [  0,   4],
       [  0,   5],
       [  0,   6],
       [  0,   7],
       [  0,   8],
       [  0,   9],
       [  0,  10],
       [  0,  11],
       [  0,  12],
       [  0,  13],
       [  0,  14],
       [  0,  15],
       [  0,  16],
       [  0,  17],
       [  0,  18],
       [  0,  19],
       [  0,  20],
       [  0,  21],
       [  0,  22],
       [  0,  23],
       [  0,  24],
       [  0,  25],
       [  0,  26],
       [  0,  27],
       [  0,  28],
       [  0,  29],
       [  0,  30],
       [  0,  31],
       [  0,  32],
       [  0,  33],
       [  0,  34],
       [  0,  35],
       [  0,  36],
       [  0,  37],
       [  0,  38],
       [  0,  39],
       [  0,  40],
       [  0,  41],
       [  0,  42],
       [  0,  43],
       [  0,  44],
       [  0,  45],
       [  0,  46],
       [  0,  47],
       [  0,  48],
       [  0,  49],
       [  0,  50],
       [  0,  51],
       [  0,  52],
       [  0,  53],
       [  0,  54],
       [  0,  55],
       [  0,  56],
       [  0,  57],
       [  0,  58],
       [  0,  59],
       [  0,  60],
       [  0,  61],
       [  0,  62],
       [  0,  63],
       [  0,  64],
       [  0,  65],
       [  0,  66],
       [  0,  67],
       [  0,  68],
       [  0,  69],
       [  0,  70],
       [  0,  71],
       [  0,  72],
       [  0,  73],
       [  0,  74],
       [  0,  75],
       [  0,  76],
       [  0,  77],
       [  0,  78],
       [  0,  79],
       [  0,  80],
       [  0,  81],
       [  0,  82],
       [  0,  83],
       [  0,  84],
       [  0,  85],
       [  0,  86],
       [  0,  87],
       [  0,  88],
       [  0,  89],
       [  0,  90],
       [  0,  91],
       [  0,  92],
       [  0,  93],
       [  0,  94],
       [  0,  95],
       [  0,  96],
       [  0,  97],
       [  0,  98],
       [  0,  99],
       [  0, 100],
       [  0, 101],
       [  0, 102],
       [  0, 103],
       [  0, 104],
       [  0, 105],
       [  0, 106],
       [  0, 107],
       [  0, 108],
       [  0, 109],
       [  0, 110]], dtype=int32), array([41, 22,  4, 36, 17,  4, 40,  6, 14,  8, 13,  6, 40, 19, 14, 40,  4,
        0, 19, 40,  8, 19, 40, 22,  4, 36, 17,  4, 40,  6, 14,  8, 13,  6,
       40, 19, 14, 40, 14, 15,  4, 13, 40, 20, 15, 40, 18, 14, 12,  4, 40,
       22,  8, 13,  4, 40,  0, 13,  3, 40, 22,  4, 36, 17,  4, 40,  6, 14,
        8, 13,  6, 40, 19, 14, 40, 19,  4, 11, 11, 40, 24, 14, 20, 40, 18,
       19, 14, 17,  0,  6,  4, 40,  7, 14, 22, 40, 12, 20,  2,  7, 40,  8,
       19, 40,  2, 14, 18, 19, 18, 40, 41], dtype=int32), array([  1, 111], dtype=int32))

Thank you in advance.

Answer 1

I have been facing the problem: The error means that

• Your 0th dimension should have a value less than 1 ie value only 0 can work.
• The 1st dimension should have values less than 7 ie values can lie only between 0 and 6.

That is why it begins crashing from indices[7] since its 1st dimension's value is 7 which is greater than 6.

Further, I suppose the problem is being caused because the number of frames(the time-step dimension) has a value which is less than the number of target_labels being sent to the ctc_loss function.

Try to make number of frames/time-steps > number of target_labels, your code should definitely work!

I would like to help further, could you please send me a link of your code.

Answer 2

It seems that ctc.loss ( https://www.tensorflow.org/api_docs/python/tf/nn/ctc_loss ) ignores the option: ignore_longer_outputs_than_inputs=True . Instead of ignoring longer outputs than input sequences it just resturns this InvalidArgumentError (it seems that I did not correctly check the lengths of my sequences).

So the solution for this is to preprocess your dataset and be sure that all the sequences that you are entering to the ctc_loss have more targets than inputs.