Libtorch：如何从 tensorRT fp16 半类型指针创建张量？

Question

how to create tensor from tensorRT fp16 half type pointer in libtorch? I am working on a detection model. I change the backbone of it to tensorRT to do FP16 inference, and the detection code such as decode boxes and nms is done in libtorch and torchvisoin, so how to create fp16 tensor from tensorRT half type pointers? The important code is to illustrate the issue:

// tensorRT code to get half type outpus
half_float::half* outputs[18];
doInference(*engine, data, outputs, 1);
// to get the final outputs with libtorch
vector<torch::Tensor> output;
//???? how to feed the date in outpus to output????
// get the result with libtorch method detect_trt->forward
 auto res = detect_trt->forward(output); 

Thanks in advance.

Answer 1

I have to do backbone inference in TensorRT, but the post process is using libtorch for convenience.And now I figure it out by using the following code:

out = torch::from_blob(outputs[i], {1, num, dim, dim}, torch::kFloat16).to(device_used);