[英]Mask RCNN OpenVino - C++ API
我想使用MaskRCNN實現自定義圖像分類器。
為了提高網絡速度,我想優化推理。
我已經使用過OpenCV DNN庫,但是我想使用OpenVINO向前邁出一步。
我成功地使用OpenVINO模型優化器(python)構建了代表我的網絡的.xml和.bin文件。
我使用Visual Studio 2017成功構建了OpenVINO Sample目錄並運行MaskRCNNDemo項目。
mask_rcnn_demo.exe -m .\Release\frozen_inference_graph.xml -i .\Release\input.jpg
InferenceEngine:
API version ............ 1.4
Build .................. 19154
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ] .\Release\input.jpg
[ INFO ] Loading plugin
API version ............ 1.5
Build .................. win_20181005
Description ....... MKLDNNPlugin
[ INFO ] Loading network files
[ INFO ] Preparing input blobs
[ WARNING ] Image is resized from (4288, 2848) to (800, 800)
[ INFO ] Batch size is 1
[ INFO ] Preparing output blobs
[ INFO ] Loading model to the plugin
[ INFO ] Start inference (1 iterations)
Average running time of one iteration: 2593.81 ms
[ INFO ] Processing output blobs
[ INFO ] Detected class 16 with probability 0.986519: [2043.3, 1104.9], [2412.87, 1436.52]
[ INFO ] Image out.png created!
[ INFO ] Execution successful
然后,我嘗試在一個單獨的項目中重現該項目。首先,我不得不觀察依賴關系。
<MaskRCNNDemo>
//References
<format_reader/> => Open CV Images, resize it and get uchar data
<ie_cpu_extension/> => CPU extension for un-managed layers (?)
//Linker
format_reader.lib => Format Reader Lib (VINO Samples Compiled)
cpu_extension.lib => CPU extension Lib (VINO Samples Compiled)
inference_engined.lib => Inference Engine lib (VINO)
opencv_world401d.lib => OpenCV Lib
libiomp5md.lib => Dependancy
... (other libs)
有了它,我就建立了一個新項目,有了自己的類和打開圖像的方式(多幀tiff)。 這項工作沒有問題,那么我將不再描述(我與CV DNN推理引擎一起使用沒有問題)。
我想構建與MaskRCNNDemo相同的項目:CustomIA
<CustomIA>
//References
None => I use my own libtiff way to open image and i resize with OpenCV
None => I will just add include to cpu_extension source code.
//Linker
opencv_world345d.lib => OpenCV 3.4.5 library
tiffd.lib => Libtiff Library
cpu_extension.lib => CPU extension compiled with sample
inference_engined.lib => Inference engine lib.
我將以下dll添加到項目目標目錄:
cpu_extension.dll
inference_engined.dll
libiomp5md.dll
mkl_tiny_omp.dll
MKLDNNPlugind.dll
opencv_world345d.dll
tiffd.dll
tiffxxd.dll
我成功編譯並執行,但是遇到兩個問題:
舊代碼:
slog::info << "Loading plugin" << slog::endl;
InferencePlugin plugin = PluginDispatcher({ FLAGS_pp, "../../../lib/intel64" , "" }).getPluginByDevice(FLAGS_d);
/** Loading default extensions **/
if (FLAGS_d.find("CPU") != std::string::npos) {
/**
* cpu_extensions library is compiled from "extension" folder containing
* custom MKLDNNPlugin layer implementations. These layers are not supported
* by mkldnn, but they can be useful for inferring custom topologies.
**/
plugin.AddExtension(std::make_shared<Extensions::Cpu::CpuExtensions>());
}
/** Printing plugin version **/
printPluginVersion(plugin, std::cout);
輸出:
[ INFO ] Loading plugin
API version ............ 1.5
Build .................. win_20181005
Description ....... MKLDNNPlugin
新代碼:
VINOEngine::VINOEngine()
{
// Loading Plugin
std::cout << std::endl;
std::cout << "[INFO] - Loading VINO Plugin..." << std::endl;
this->plugin= PluginDispatcher({ "", "../../../lib/intel64" , "" }).getPluginByDevice("CPU");
this->plugin.AddExtension(std::make_shared<Extensions::Cpu::CpuExtensions>());
printPluginVersion(this->plugin, std::cout);
輸出:
[INFO] - Loading VINO Plugin...
000001A242280A18 // Like memory adress ???
第二期:
當我嘗試從新代碼中提取投資回報率和蒙版時,如果我有一個“匹配項”,我總是會:
但是面膜看起來很好提取...
新代碼:
float score = box_info[2];
if (score > this->Conf_Threshold)
{
// On reconstruit les coordonnées de la box..
float x1 = std::min(std::max(0.0f, box_info[3] * Image.cols), static_cast<float>(Image.cols));
float y1 = std::min(std::max(0.0f, box_info[4] * Image.rows), static_cast<float>(Image.rows));
float x2 = std::min(std::max(0.0f, box_info[5] * Image.cols), static_cast<float>(Image.cols));
float y2 = std::min(std::max(0.0f, box_info[6] * Image.rows), static_cast<float>(Image.rows));
int box_width = std::min(static_cast<int>(std::max(0.0f, x2 - x1)), Image.cols);
int box_height = std::min(static_cast<int>(std::max(0.0f, y2 - y1)), Image.rows);
Image is resized from (4288, 2848) to (800, 800)
Detected class 62 with probability 1: [4288, 0], [4288, 0]
然后當我沒有正確的bbox坐標時,我不可能將蒙版放置在圖像中並調整其大小...
有人對我做得不好有想法嗎?
如何使用cpu_extension正確創建和鏈接OpenVINO項目?
謝謝 !
版本的第一個問題:在printPluginVersion函數上方,您將看到InferenceEngine和插件版本信息的重載std :: ostream運算符。
第二:您可以嘗試通過對原始框架和OV進行第一個卷積和輸出層比較后的輸出來調試模型。 確保每個元素都相等。
在OV中,您可以使用network.addOutput(“ layer_name”)將任何圖層添加到輸出中。 然后使用以下命令讀取輸出:const Blob :: Ptr debug_blob = infer_request.GetBlob(“ layer_name”)。
大多數時候,我會遇到諸如此類的問題,我發現缺少輸入預處理(均值,規范化等)。
cpu_extensions是一個動態庫,但是您仍然可以更改cmake腳本以使其靜態並與應用程序鏈接。 之后,您需要將應用程序路徑與對IExtensionPtr的調用一起使用extension_ptr = make_so_pointer(argv [0])
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.