Issues using tensorflow-gpu 1.7.0

Question

I want to test a model proposed here for that I need tensorflow-gpu version 1.7.0, I installed the required version using pip command pip install tensorflow-gpu==1.7.0 everything runs, but when I run the training I get the following error

====NCBI_MODEL Training=====
Traceback (most recent call last):
  File "run.py", line 226, in <module>
    sess, batch_idx, infoInput=intOuts, tbWriter=tbWriter)
  File "/home/youssef/Documents/collabonet/CollaboNet-master/model/RunModel.py", line 82, in train1epoch
    feed_dict=feed_dict1)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 905, in run
    run_metadata_ptr)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1140, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
    run_metadata)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[2,15] = 1651 is not in [0, 2)
     [[Node: BC5CDR-diseaseWE/embedding_lookup = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@BC5CDR-diseaseWE/embed"], validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](BC5CDR-diseaseWE/embed, _arg_X_0_0)]]

Caused by op u'BC5CDR-diseaseWE/embedding_lookup', defined at:
  File "run.py", line 119, in <module>
    modelDict[dataSet]['WE']=modelClass.we(args=modelDict[dataSet]['args'])
  File "/home/youssef/Documents/collabonet/CollaboNet-master/model/CollaboNet.py", line 104, in we
    reuse=False)
  File "/home/youssef/Documents/collabonet/CollaboNet-master/ops/embeddingOps.py", line 76, in embedding_lookup
    inputs_embed = tf.nn.embedding_lookup(embedding_table, inputs)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/ops/embedding_ops.py", line 327, in embedding_lookup
    transform_fn=None)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/ops/embedding_ops.py", line 151, in _embedding_lookup_and_transform
    result = _clip(_gather(params[0], ids, name=name), ids, max_norm)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/ops/embedding_ops.py", line 55, in _gather
    return array_ops.gather(params, ids, name=name)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 2698, in gather
    params, indices, validate_indices=validate_indices, name=name)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2672, in gather
    validate_indices=validate_indices, name=name)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3290, in create_op
    op_def=op_def)
  File "/home/youssef/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1654, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): indices[2,15] = 1651 is not in [0, 2)
     [[Node: BC5CDR-diseaseWE/embedding_lookup = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@BC5CDR-diseaseWE/embed"], validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](BC5CDR-diseaseWE/embed, _arg_X_0_0)]]

I suspected that it has something to do with tensorflow version, so when I run import tensorflow as tf tf.__version__ the output is '1.7.0' I concluded that the version I got is the cpu version, however I tried to uninstall it and reinstall it the same problem presists. I have also tried the tensorflow-gpu 1.7.0 docker image but it gives the same outputs. So is there any way I can surpass this problem?

Answer 1

It seems like the problem was caused by an empty file which in consequence resulted in an empty feed dictionary.