Read multi-dimensional array from caffe in opencv (C++)

Question

I have a model in caffe that produce a multi-dimensional array. Size of this array is [1x10x8x8] so in python I haven't problem with this size because python automatically manage this array and I know order of elements in that. but when I switch to opencv with c++ the whole array is a vector and I haven't any idea how to regenerate something like python array, I use cv::NAryMatIterator to access multi-dimensional array such below

const cv::Mat* arrays[]={&prob,0};  //my multi-dimensional array is prob
cv::Mat my_planes[1];
cv::NAryMatIterator it(arrays,my_planes);
cv::Mat Multi_Array ;                               //temporary Mat 
for (int p = 0; p < it.nplanes; ++p,++it) {
    Multi_Array = it.planes[0];
}

after doing that I see Multi_Array size is [640x1] which seems that it is equal to 8x8x10 that python produced. is there anyway to access 8x8 planes one by one?

EDIT: my multi-dimensional array size is [1x10x8x8]

Answer 1

To access the 3D array as if it were a 2D array with shape [640][1] , you could write 3 loops to iterate on the elements using a [x,y,z] format like:

int data[640][1] = { 0 };
int width = 8, height = 8, depth = 10;

for (int x = 0; x < width; x++)
    for (int y = 0; y < height; y++)
        for (int z = 0; z < depth; z++)
        {
            int idx = x * height * depth + y * depth + z;
            data[idx][0] = idx;
        }

This fills the array with numbers ranging from 0 to 639.

If you are looking to access a 2D array as a 1D, check this answer .

Answer 2

If your model data is ordered in row-major form, you can have OpenCV interpret the data as a Mat of the required size. Then, planes of the Mat can be accessed using multidim_mat.row( row_number ) .

In order to create a Mat from the data:

int data[640] = { 0 };
const int size[] = { 8, 8, 10 };

cv::Mat multidim_mat(3, size, CV_32S, data);

std::cout << multidim_mat.dims << std::endl;
for (int i = 0; i < multidim_mat.dims; i++) {
    std::cout << "Dimension " << i << " is of size " << multidim_mat.size[i] << std::endl;
}

The CV_32S is to inform OpenCV to interpret the data as signed 32-bit integers.

References: https://docs.opencv.org/3.4.0/d3/d63/classcv_1_1Mat.html#a5fafc033e089143062fd31015b5d0f40 , https://docs.opencv.org/3.4.0/d3/d63/classcv_1_1Mat.html#details ,

Answer 3

In first step we need to get a pointer to OpenCV Mat object, you can do this by the below command.(I assume that data that represent your data is primarily float and consider probability Mat is prob which we get this Mat from caffe)

float* p = (float*)(prob.data);

This pointer will points to the where data is reside in memory. So for example if we want to get access to the element in (1,3,7,7) location we can do this operation like this:

int S= sizeof(float);    
float val = p[(
    7*p.step[3]/S +    //forth dimension
    7*p.step[2]/S +    //third dimension
    3*p.step[1]/S      //second dimension
  )]           
 //first dimension is not needed, because it is decoded in address of p
 //and if you have any higher number than 1 in first dimension you need to add it to the above command

So for traversing in the probability matrix you can do that like the below:

auto S=sizeof(float);
for (int d2 = 0; d2 < 129; ++d2) {
    for (int d3 = 0; d3 < 129; ++d3) {
        for (int d4 = 0; d4 < 10; ++d4) {
            float val = p[(d2*prob.step[3]/S + d3*prob.step[2]/S + d4* prob.step[1]/S)];
        }
    }
}