why tensorflow can't connect to hdfs?

Question

I want to connect hdfs with tensorflow,but it can't. I debug into the tensorflow code found error as followed:

(Pdb) s
--Return--
> /usr/local/python3/lib/python3.5/site-packages/tensorflow/python/util/compat.py(60)as_bytes()->b'hdfs://user...c/mnist_logs3'
-> return bytes_or_text.encode(encoding)
(Pdb) l
55  
56     Raises:
57       TypeError: If `bytes_or_text` is not a binary or unicode string.
58     """
59     if isinstance(bytes_or_text, _six.text_type):
60  ->     return bytes_or_text.encode(encoding)
61     elif isinstance(bytes_or_text, bytes):
62       return bytes_or_text
63     else:
64       raise TypeError('Expected binary or unicode string, got %r' %
65                       (bytes_or_text,))
(Pdb) s
loadFileSystems error:
(unable to get stack trace for java.lang.NoClassDefFoundError exception: ExceptionUtils::getStackTrace error.)
hdfsBuilderConnect(forceNewInstance=0, nn=user, port=0, kerbTicketCachePath=(NULL), userName=(NULL)) error:
(unable to get stack trace for java.lang.NoClassDefFoundError exception: ExceptionUtils::getStackTrace error.)
> /usr/local/python3/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py(395)is_directory()
-> pywrap_tensorflow.TF_DeleteStatus(status)
(Pdb)

can someone help me? Thanks!

Answer 1

My best guess is it's failing to load the appropriate library. Double-check that you have Hadoop installed on your machine and set the environment variables appropriately. See the instructions here: https://www.tensorflow.org/deploy/hadoop