Indexing tensorflow output tensor in c++

Question

I am loading a graph (*.pb) using the C++ API. The graph has been set up and trained in Python with an input shape definition of the graph: tf.placeholder(tf.float32, [None, 84, 84, 1], name='in' . This should allow to feet an arbitrary batch size. After starting a session and loading the graph I take a rectangular greyscale OpenCV Mat image and I split it in smaller square images, resize them to the needed input size and store them in a vector:

cv::Size smallSize(splitLength, img_in.size().height);
std::vector<Mat> input_Images;
int y = 0;
for (int x = 0; x < img_in.cols; x += smallSize.width)
{
    cv::Rect rect =  cv::Rect(x,y, smallSize.width, smallSize.height);
    cv::Mat temp = cv::Mat(img_in, rect);
    cv::Size s(height_out, width_out);
    cv::resize(temp,process_img,s,0,0,cv::INTER_CUBIC);
    input_Images.push_back(process_img);
}

Then I write this array to a tensorflow tensor:

tensorflow::Tensor input_tensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({input_Images.size(), height_out, width_out, 1}));
auto input_tensor_mapped = input_tensor.tensor<float, 4>();

for (int i = 0; i < input_Images.size(); i++) {
    Mat image = input_Images[i];
    const float * source_data = (float*) image.data;
    for (int h = 0; h < image.rows; ++h) {
        const float* source_row = source_data + (h * image.cols * image.channels());
        for (int w = 0; w < image.cols; ++w) {
            const float* source_pixel = source_row + (w * image.channels());
            for (int c = 0; c < image.channels(); ++c) {
                const float* source_value = source_pixel + c;
                input_tensor_mapped(i, h, w, c) = *source_value;
            }
        }
    }
}

I get a tensor with the shape of [16,84,84,1]. Then I run the session :

session_create_status = session_deepcytometry->Run({{ inputLayer, nn_input_tensor}},{outputLayer},{},&finalOutput);

This seems to work just fine. When I run std::cout finalOutput[0].DebugString() << "\\n"; I get as output: stringTensor<type: float shape: [16,4] values: [7.8605752 10.652889 -24.507538]...>

In the case of batch size 1 it shows me: stringTensor<type: float shape: [1,4] values: [7.8605752 10.652889 -24.507538]...>

finalOutput.size(); is in either case 1.

If the batch size is 1 I retrieve the class scores with the simple loop:

for(int i=0; i<nClasses; i++){
        result.push_back(finalOutput[0].flat<float>()(i));
    }

The question is now how I do this if the batch size is 16?

Answer 1

You should access the tensor like in the beginning. If the output shape has rank 2, then use

auto finalOutputTensor  = finalOutput[0].tensor<float, 2>();

And

for(int b=0; b<BatchSize;b++)
for(int i=0; i<nClasses; i++){
    cout << b << "th output for class "<<i<<" is "<< finalOutputTensor(b, i) <<end; 
}

In your case of handling a flat tensor (as an equivalent alternative) you could also use

for(int b=0; b<BatchSize;b++)
for(int i=0; i<nClasses; i++){
    cout << b << "th output for class "<<i<<" is "<< finalOutput[0].flat<float>()(b * nClasses + i) << end; 
}