拆分 ONNX DNN 模型

Question

I'm trying to split DNN Models in order to execute part of the network on the edge and the rest on the cloud . Because it has to be cross-platform and work with every framework I need to do it directly starting from an ONNX model .

I know how to generate an ONNX model starting from tensorflow/keras and how to run an ONNX model, but I realized that is really hard to work on the ONNX file, like visualizing it and modify it.

Is there someone that can help me understand how to split and ONNX model, or at least run part of an ONNX model (like from input to layer N and from layer N to the output)?

I'm starting from this situation:

# load MobileNetV2 model
model = MobileNetV2()

# Export the model
tf.saved_model.save(model, "saved_model")  

# export to .onnx
!python -m tf2onnx.convert --saved-model saved_model --output mobilenet_v2.onnx --opset 7

# open the saved ONNX Model
print("Import ONNX Model..")
onnx_model = onnx.load("mobilenet_v2.onnx")
tf_rep = prepare(onnx_model, logging_level="WARN", auto_cast=True)

I tried to use sclblonnx but on models this big(although it's a small model) I can't really print the graph and when I list the inputs and outputs with textlist_inputs/list_outputs I don't really get how ther are interconnected.

Any help would be greatly appreciated. Thank you in advance.

Answer 1

从Onnx PythonAPI 规范中，您可以通过指定张量的输入名称和输出名称来拆分 onnx 模型。

Answer 2

The first thing you probably need to do is understand the underlining graph for the onnx model you have.

onnx_graph = onnx_model.graph

Will return the graph object.

After that, you need to understand where you want to separate this graph into two separate graphs (and so run two models).

You can plot the graph with Netron (this is what sclblonnx does) or you can try to look inside manually by looking at

onnx_graph_nodes = onnx_graph.node

Of course looking at the graph inputs( onnx_graph.input ) and outputs ( onnx_graph.output ) is also important.

If you look at the " merge " file from sclblonnx you will see the syntax details for diving into a graph as well as a "split" function at may help you.