[英]Error converting FaceNet model into ONNX format
系統信息
問題
我從這個頁面下載了一個 tensorflow model 的 FaceNet,我試圖將它從 .pb 轉換成一個 .onnx 文件,但是它引發了以下錯誤:
重現
root@xesk-VirtualBox:/home/xesk/Desktop# python -m tf2onnx.convert --saved-model home/xesk/Desktop/2s/20180402-114759/20180402-114759.pb --output model.onnx
2020-08-03 20:18:05.081538: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2020-08-03 20:18:05.081680: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2020-08-03 20:18:07,431 - WARNING - '--tag' not specified for saved_model. Using --tag serve
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.8/dist-packages/tf2onnx/convert.py", line 171, in
main()
File "/usr/local/lib/python3.8/dist-packages/tf2onnx/convert.py", line 131, in main
graph_def, inputs, outputs = tf_loader.from_saved_model(
File "/usr/local/lib/python3.8/dist-packages/tf2onnx/tf_loader.py", line 288, in from_saved_model
_from_saved_model_v2(model_path, input_names, output_names, tag, signatures, concrete_function)
File "/usr/local/lib/python3.8/dist-packages/tf2onnx/tf_loader.py", line 247, in _from_saved_model_v2
imported = tf.saved_model.load(model_path, tags=tag) # pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 603, in load
return load_internal(export_dir, tags, options)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/load.py", line 614, in load_internal
loader_impl.parse_saved_model_with_debug_info(export_dir))
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/loader_impl.py", line 56, in parse_saved_model_with_debug_info
saved_model = _parse_saved_model(export_dir)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/saved_model/loader_impl.py", line 110, in parse_saved_model
raise IOError("SavedModel file does not exist at: %s/{%s|%s}" %
OSError: SavedModel file does not exist at: home/xesk/Desktop/2s/20180402-114759/20180402-114759.pb/{saved_model.pbtxt|saved_model.pb}
附加上下文
我沒有運行任何 CUDA 或類似的東西,只有 CPU。 下載的model是20180402-114759 。 這是我第一次使用這些工具,而且我在這個 AI 世界中還是個初學者,所以我可能遺漏了一些明顯的東西。 當然,我多次檢查了路徑和命令語法。 可能與我下載的文件格式有關?
編輯
根據Venkatesh Wadawadagi的回答,我選擇了選項 1。更改.meta文件的名稱解決了腳本無法識別的問題。
該腳本或多或少正確運行,並完成創建 export_dir 目錄,其中export_dir > 0 > variables子文件夾。 然而,它們是空的。
控制台output是這樣的:
xesk@xesk:~/Desktop/UP2S/ACROMEGALLY/20180402-114759$ python3 ./pb2sm
2020-08-10 16:02:26.128846: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-08-10 16:02:26.129114: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2020-08-10 16:02:26.129137: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (xesk): /proc/driver/nvidia/version does not exist
2020-08-10 16:02:26.129501: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-08-10 16:02:26.139076: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2592000000 Hz
2020-08-10 16:02:26.139506: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x44018d0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-10 16:02:26.139520: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/queue_runner_impl.py:391: QueueRunner.__init__ (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.
2020-08-10 16:02:32.681265: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 17676288 exceeds 10% of system memory.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value InceptionResnetV1/Block8/Branch_0/Conv2d_1x1/BatchNorm/beta/Adam
[[{{node save/SaveV2_1}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./pb2sm", line 17, in <module>
strip_default_attrs=True)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/saved_model/builder_impl.py", line 595, in add_meta_graph_and_variables
saver.save(sess, variables_path, write_meta_graph=False, write_state=False)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 1193, in save
raise exc
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 1176, in save
{self.saver_def.filename_tensor_name: checkpoint_file})
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value InceptionResnetV1/Block8/Branch_0/Conv2d_1x1/BatchNorm/beta/Adam
[[node save/SaveV2_1 (defined at /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
Original stack trace for 'save/SaveV2_1':
File "./pb2sm", line 17, in <module>
strip_default_attrs=True)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/saved_model/builder_impl.py", line 589, in add_meta_graph_and_variables
saver = self._maybe_create_saver(saver)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/saved_model/builder_impl.py", line 227, in _maybe_create_saver
allow_empty=True)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 828, in __init__
self.build()
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 840, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 878, in _build
build_restore=build_restore)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 499, in _build_internal
save_tensor = self._AddShardedSaveOps(filename_tensor, per_device)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 291, in _AddShardedSaveOps
return self._AddShardedSaveOpsForV2(filename_tensor, per_device)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 265, in _AddShardedSaveOpsForV2
sharded_saves.append(self._AddSaveOps(sharded_filename, saveables))
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 206, in _AddSaveOps
save = self.save_op(filename_tensor, saveables)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/training/saver.py", line 122, in save_op
tensors)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/ops/gen_io_ops.py", line 1946, in save_v2
name=name)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_core/python/framework/ops.py", line 1748, in __init__
self._traceback = tf_stack.extract_stack()
有沒有可能我缺少一些要安裝的庫? 似乎與某些 CUDA 實現有關,但我沒有。 可能嗎?
您正在使用的命令:
python -m tf2onnx.convert --saved-model home/xesk/Desktop/2s/20180402-114759/20180402-114759.pb --output model.onnx
請注意,您正在使用的Fac.net trained model 只有凍結圖( .pb
文件)和檢查點( .ckpt
)並且沒有您的命令正在尋找的saved-model
。
所以基本上你將路徑傳遞給凍結圖的.pb
文件,這與SavedModel的.pb
文件(你沒有)不同。 Savedmodel 將包含variables
文件夾和saved_model.pb
文件。
這就是錯誤的原因:
OSError: SavedModel file does not exist
在此處閱讀有關 SavedModel 的更多信息。
要繼續進行 ONNX 轉換,您有兩種選擇:
為此使用以下代碼:
import os
import tensorflow as tf
trained_checkpoint_prefix = 'model-20180402-114759.ckpt-275'
export_dir = os.path.join('export_dir', '0')
graph = tf.Graph()
with tf.compat.v1.Session(graph=graph) as sess:
# Restore from checkpoint
loader = tf.compat.v1.train.import_meta_graph(trained_checkpoint_prefix + '.meta')
loader.restore(sess, trained_checkpoint_prefix)
# Export checkpoint to SavedModel
builder = tf.compat.v1.saved_model.builder.SavedModelBuilder(export_dir)
builder.add_meta_graph_and_variables(sess,
[tf.saved_model.TRAINING, tf.saved_model.SERVING],
strip_default_attrs=True)
builder.save()
注意: .data
、 .index
和.meta
應該有相同的前綴,然后這段代碼才能工作。 所以重命名.meta
文件。
mv model-20180402-114759.meta model-20180402-114759.ckpt-275.meta
ckpt
文件或frozen-graph.pb
進行 onnx 轉換從檢查點格式:
python -m tf2onnx.convert --checkpoint tensorflow-model-meta-file-path --output model.onnx --inputs input0:0,input1:0 --outputs output0:0
來自 graphdef/frozen-graph 格式:
python -m tf2onnx.convert --graphdef tensorflow-model-graphdef-file --output model.onnx --inputs input0:0,input1:0 --outputs output0:0
如果您的 TensorFlow model 的格式不是saved model
,那么您需要提供 model 圖的inputs
和outputs
。
從這個:
如果您的 model 是檢查點或graphdef格式,並且您不知道 model 的輸入和 output 節點,則可以使用summarize_graph TensorFlow 實用程序。 summarize_graph工具確實需要從源代碼下載和構建。 如果您可以選擇前往您的 model 提供商並以保存的 model 格式獲取 model,那么我們建議您這樣做。
我遇到過類似的錯誤。 在我的例子中,我錯誤地給出了 pb 文件而不是path/to/savedmodel ,它應該是包含saved_model.pb的目錄的路徑。 因此,假設您的20180402-114759.pb
位於目錄home/xesk/Desktop/2s/20180402-114759
中,命令應為:
python -m tf2onnx.convert --saved-model home/xesk/Desktop/2s/20180402-114759 --output model.onnx
有關詳細信息,請參閱開始將 TensorFlow 轉換為 ONNX和使用 SavedModel 格式。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.