简体繁体 English

如何使用 C/C++ 对训练有素的神经网络进行推理？

[英]How can I do the inference for a trained neural network using C/C++?

原文 2019-04-12 12:47:16 8 1 c/ tensorflow/ machine-learning/ neural-network/ deep-learning

For my master's thesis, I will have to do an inference with a pre-built / pre-trained (with TensorFlow) deep neural network model.对于我的硕士论文，我将不得不使用预构建/预训练（使用 TensorFlow）深度神经网络模型进行推理。 I received it in two different formats (hdf5 / h5 and frozen graph =.pb).我收到了两种不同格式的文件（hdf5 / h5 和 frozen graph =.pb）。 The inference shall be done on a cluster, so far we only have a GPU-version (with TensorRT and a uff Model) running.推理应在集群上完成，到目前为止，我们只有一个 GPU 版本（带有 TensorRT 和一个 uff 模型）在运行。 So my first job seems to be to do inference on one CPU before making a usage possible on the cluster.所以我的第一份工作似乎是在集群上使用之前在一个 CPU 上进行推理。

We are using the model within computational fluid dynamics (CFD) simulations – that is also my academic background, and as you can therefore imagine I have only a little knowledge about deep learning.我们在计算流体动力学 (CFD) 模拟中使用该模型——这也是我的学术背景，因此你可以想象我对深度学习知之甚少。 Anyway, it is not my job to change/train the model but just to use it for inference.无论如何，更改/训练模型不是我的工作，而只是将其用于推理。 Our CFD-Code is written in C++, which is the only programming language I am using on an advanced level (obviously it is no problem to use C, but I have no idea of python).我们的 CFD 代码是用 C++ 编写的，这是我在高级水平上使用的唯一编程语言（显然使用 C 没问题，但我不知道 python）。

After going through many Google searches I recognized that I do not have a real idea how to start things off.经过多次谷歌搜索后，我意识到我并不知道如何开始。 I thought it would be possible to skip all the training and TensorFlow stuff.我认为可以跳过所有训练和 TensorFlow 的东西。 I know how neural networks work and how they calculate their output values from their input values.我知道神经网络如何工作以及它们如何根据输入值计算输出值。 I also have the most important theoretical knowledge, but no programming knowledge in this field.我也有最重要的理论知识，但没有这方面的编程知识。 Is it somehow possible to use the model they gave me (so either hdf5/h5 or frozen graph) and build an inference code using exclusively C or C++?是否有可能以某种方式使用他们给我的模型（hdf5/h5 或冻结图）并仅使用 C 或 C++ 构建推理代码？ I already found the C API and installed it within a docker container (where I also have Tensorflow), but I am really not sure what the next step is.我已经找到了 C API 并将其安装在 docker 容器中（我也有 Tensorflow），但我真的不确定下一步是什么。 What can I do with the C API?我可以用 C API 做什么？ How would you write a C/C++-Code for inference with a DNN-model that is prepared to inference with it?您将如何编写 C/C++ 代码以使用准备好对其进行推理的 DNN 模型进行推理？

1 个解决方案

Opencv provided tools to run deep learning models but they are just limited to computer vision field. Opencv 提供了运行深度学习模型的工具，但它们仅限于计算机视觉领域。 See here .看这里。

You can perform classification, object detection, face detection, text detection, segmentation, and so on by using the API provided by opencv.您可以使用opencv提供的API进行分类、物体检测、人脸检测、文本检测、分割等。 These examples are fairly straightforward.这些示例相当简单。

There are both python version and c++ version available.有python版本和c++版本可用。