tensorflow java模型推断将获取的张量转换为字符串？

Question

I'm now using tensorflow(python) to train my models, and want to use tensorflow(java) to inference result online.

computation graph has an operation to return shape[1,16] result, each element in the tensor is a string. Now I want to convert the result into whole string.

I create a ByteBuffer, and call Tensor.writeTo to write data in the buffer. But when I decode the final buffer, it has some unexpected chars in the headings, I guess the final bytes may include some tensor meta information.

Tensor predictedTensor = result.get(0);
ByteBuffer bb = ByteBuffer.allocate(predictedTensor.numBytes());
predictedTensor.writeTo(bb);
String predictedTokens = null;
byte[] bbArray = bb.array();
predictedTokens = new String(bbArray, "UTF-8");

and the result is like this: first part is some incorrect codes, the last part is right.

& *  ？ *  C J M X & *  ？ *  C J M X hello,world!

I guess maybe the Tensor with shape(1,16) has meta information in the bytes, but I don't how to fetch what I need part.

Can anyone know how to convert a multi-dimensional tensor into java string in java tensorflow interface?

Answer 1

I find a workaroud for this! When train the model, I call tf.reduce_join on the tensor with shape(1,16) to get one scalar. And when inference in java language, I just fetch that scalar node, and call tensor.byteValue() to get tensor bytes. It will return pure result without heading codes.

Answer 2

If the result of the operation has shape [1, 16] , then it means it is producing 16 different strings, not one string.

Support for multi-dimensional string tensors in Java was added only recently ( github commit ) and is not included in the pre-built binaries for TensorFlow release 1.3 and before. You will have to either build from source or wait for the TensorFlow 1.4 release for that.

With that feature you should be able to decode your (1, 16) shaped tensor with something like this:

Tensor predictedTensor = result.get(0);
byte[][][] predictedTokenBytes = predictedTensor.copyTo(new byte[1][16][]);
String[] predictedTokens = new String[16];
for (int i = 0; i < 16; ++i) {
  // This works under the assumption that the model is actually
  // producing UTF-8 strings    
  predictedTokens[i] = new String(predictedTokenBytes[0][i], "UTF-8");
}

If you really need a single string, then yeah, you could use tf.reduce_join to have the model combine the 16 strings into one and then extract the scalar.