如何在 C++ 中将 TF_Tensor 转换为 opencv Mat？

Question

I am trying to port Python Tensorflow model to C++. In process, I need to convert the TF_Tensor class to cv::Mat.

I created the output tensor as the below.

TF_Tensor** OutputValues = (TF_Tensor**)malloc(sizeof(TF_Tensor*) * NumOutputs);

Then I loaded model and the session was completed successfully, but I failed to convert OutputValues to cv::Mat.

I obtained a pointer to the data buffer by the code below.

const float* camBuf = (float*)TF_TensorData(*OutputValues);

But when I tried to create cv::Mat by the code below,

cv::Mat testInputImage(
        80,
        80,
        3,
        TF_TensorData(*OutputValues)
    );

Image is not generated correctly.

I could not find any reference to TF_Tensor data structure, so I am asking for a help.

Answer 1

try :

cv::Mat mat(width, height, CV_32F);
std::memcpy((void *)mat.data, camBuf , sizeof(TF_Tensor*) * NumOutputs);

Answer 2

By doing:

cv::Mat testInputImage(80, 80, CV_32FC(3), TF_TensorData(*OutputValues));

you are "wrapping" the existing data in a cv::Mat , which avoids a copy. Note that the third argument should be CV_32FC(3) (a 32-bit floating point image, with 3 channels). This approach should work if OutputValues is a TF_Tensor** type, and if the underlying TF_Tensor holds appropriate data.

However, I don't think that this:

TF_Tensor** OutputValues = (TF_Tensor**)malloc(sizeof(TF_Tensor*) * NumOutputs);

is an appropriate way to allocate a TF_Tensor; I think you should be using TF_AllocateTensor instead.

All that said, if you are using C++, you might consider using tf::Tensor API instead of TF_Tensor (which is used for C, and is less common).

You omitted some details, but let's say your tensor is 4-dimensional (as is common), has float32 values, and is laid out as NxHxWxC (In other words, the tensor is holding a collection of float images). If you want to convert idx -th element in the batch to a cv::Mat , you can do it like this:

tf::Tensor tensor = /* tensor from somewhere */;
int idx = /* index of the image in the batch */;

int batch_size = tensor.dim_size(0);
int rows = tensor.dim_size(1);                                                
int cols = tensor.dim_size(2);                                                
int channels = tensor.dim_size(3);                                            
int row_size = channels * cols * sizeof(float);                               
                                                                                
cv::Mat image(rows, cols, CV_32FC(channels));                                
auto tensor_mapped = tensor.tensor<float, 4>();                       
for (int r = 0; r < rows; ++r) {                                              
  float* row = reinterpret_cast<float*>(mat.data + r * row_size);            
  for (int c = 0; c < cols; ++c) {                                  
    for (int k = 0; k < channels; ++k) {                                      
      row[k + c * channels] = tensor_mapped(idx, r, c, k);                    
    }                                                                         
  }                                                                           
}