如何将 opencv mat 灰度图像转换为 pytorch 张量？

Question

I want to convert cv::Mat class into torch::Tensor class and put them into CNN module of libtorch for prediction. How should I convert Mat class to Tensor class? My CNN module is a pt file generated using the mnist training set. The training image of this mnist training set is single-channel gray image. If I want to test, it should load a gray mat image, so I want to change the single-channel gray image converted to tensor. The language I use is c++。

Answer 1

The idea is to copy memory from cv::Mat object to torch::Tensor object:

    cv::Mat img =cv::imread("path/to/file.png");
    cv::cvtColor(img,img,cv::COLOR_BGR2GRAY);
    torch::Tensor tensor = torch::empty({img.cols,img.rows},torch::TensorOptions().dtype(torch::kUInt8));
    memcpy(tensor.data<void>(),img.data,img.rows*img.cols);

Answer 2

Thank you for your help. Finally, I found a solution. Before, I didn't know much about the tensor class. A tensor has four elements, which are the number, height, width, and byte depth of the image.

  std::string p1 = "D:";
  std::string p2;
  std::cin >> p2;
  std::string p3 = ".png";
  std::string path = p1 + p2 + p3;
  cv::Mat image = cv::imread(path,cv::IMREAD_GRAYSCALE);
  torch::Tensor img_tensor = torch::from_blob(image.data, { 1, image.rows, image.cols, 1 }, torch::kByte);
  img_tensor = img_tensor.permute({ 0, 3, 1, 2 });
  img_tensor = img_tensor.toType(torch::kFloat);
  img_tensor = img_tensor.div(255);
  auto a = net->predict(img_tensor);
  std::cout << a.argmax() << std::endl;

Answer 3

I use this combination for convert openCV Images to tensor of libtorch. I use libtorch-cxx11-abi-shared-with-deps-1.10.0+cu102 and C++ STANDARD 14.

cv::Mat image = cv::imread("test.jpeg")// Any image to load...
cv::Mat input; // Conversion of image in to color format or any format [1].
cv::cvtColor(image, input,  cv::COLOR_RGBGRAY); 
//cv::cvtColor(image, input, cv::COLOR_BGR2RGB);    //or for color  
                               
std::cout << "Check image dimentions" <<std::endl;  // Check conversion
std::cout << "pixel dim x: "<<input.rows<< std::endl;
std::cout << "pixel dim x: "<<input.cols<< std::endl;
//Image to tensor conversion.
torch::Tensor tensor_image = torch::from_blob(input.data, {1, input.rows, input.cols, 3}, torch::kByte);
//Permute data structure to Tensor format.
tensor_image = tensor_image.permute({0, 3, 1, 2});
tensor_image = tensor_image.toType(torch::kFloat);
tensor_image = tensor_image.div(255);
//load to CUDA device if need...
tensor_image = tensor_image.to(torch::kCUDA);
std::cout<<"Img to Tensor converted!"<<std::endl;

[1] https://docs.opencv.org/3.4/d8/d01/group__imgproc__color__conversions.html