使用 load_model 加载经过 model 训练的 tensorflow.keras 返回 JSON 解码错误，而未经训练的 model 加载正常

Question

I have a trained Keras model built and trained using the tensorflow.keras API and saved using the tf.keras.save_model() method with no optional arguments. Tensorflow is up to date and my Python version is 3.8. 据我了解，此方法应使用 TF 2.X 中推荐的默认“tf”格式保存 model，然后使用load_model()应该可以正常工作。

但是，再次加载 model 会产生以下内容：

model = tf.keras.models.load_model("/Volumes/thesis_drive/thesis_project_local_new/trained_model_640x64/")

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
/var/folders/c1/8tq0t8y90195qxyt5hppjjtr0000gq/T/ipykernel_933/1131710361.py in <module>
----> 1 model = tf.keras.models.load_model("/Volumes/thesis_drive/thesis_project_local_new/trained_model_640x64/")

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/save.py in load_model(filepath, custom_objects, compile, options)
    204         filepath = path_to_string(filepath)
    205         if isinstance(filepath, str):
--> 206           return saved_model_load.load(filepath, compile, options)
    207 
    208   raise IOError(

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/load.py in load(path, compile, options)
    153 
    154   # Finalize the loaded layers and remove the extra tracked dependencies.
--> 155   keras_loader.finalize_objects()
    156   keras_loader.del_tracking()
    157 

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/load.py in finalize_objects(self)
    624 
    625     # Initialize graph networks, now that layer dependencies have been resolved.
--> 626     self._reconstruct_all_models()
    627 
    628   def _unblock_model_reconstruction(self, layer_id, layer):

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/load.py in _reconstruct_all_models(self)
    643       all_initialized_models.add(model_id)
    644       model, layers = self.model_layer_dependencies[model_id]
--> 645       self._reconstruct_model(model_id, model, layers)
    646       _finalize_config_layers([model])
    647 

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/load.py in _reconstruct_model(self, model_id, model, layers)
    659   def _reconstruct_model(self, model_id, model, layers):
    660     """Reconstructs the network structure."""
--> 661     config = json_utils.decode(
    662         self._proto.nodes[model_id].user_object.metadata)['config']
    663 

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/json_utils.py in decode(json_string)
     60 
     61 def decode(json_string):
---> 62   return json.loads(json_string, object_hook=_decode_helper)
     63 
     64 

~/miniforge3/lib/python3.9/json/__init__.py in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    357     if parse_constant is not None:
    358         kw['parse_constant'] = parse_constant
--> 359     return cls(**kw).decode(s)

~/miniforge3/lib/python3.9/json/decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

~/miniforge3/lib/python3.9/json/decoder.py in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

为了测试这是否是save_model()或load_model()的错误，我在 Jupyter notebook 中再次构建了相同的 model，保存并重新加载，没有出现错误：

import tensorflow as tf
from tensorflow.keras.layers import Dense, Activation, Flatten, Dropout, Conv2D, MaxPooling2D, Input
from tensorflow.keras.losses import CategoricalCrossentropy
from tensorflow.keras import optimizers
from tensorflow.keras.models import Model

def build_model():
    _input = Input(shape=(640,64,3))
    x = Conv2D(filters=64, kernel_size=4, input_shape=(640, 64, 3))(_input)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(4, 4))(x)
    x = Dropout(0.5)(x)
    x = Conv2D(filters=128, kernel_size=4, input_shape=(640, 64, 3))(_input)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(4, 4))(x)
    x = Dropout(0.5)(x)
    x = Conv2D(filters=256, kernel_size=4, input_shape=(640, 64, 3))(_input)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(2, 2))(x)
    x = Dropout(0.5)(x)
    x = Flatten()(x)
    output = Dense(161, activation = 'softmax')(x)
    model = Model(_input,output)
    model.compile(optimizer=optimizers.Adam(), loss="categorical_crossentropy")
    tf.keras.models.save_model(model,"model_test")

model = build_model()

Metal device set to: Apple M1
2021-09-23 13:40:46.234438: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2021-09-23 13:40:46.234631: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2021-09-23 13:40:47.112730: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
INFO:tensorflow:Assets written to: model_test/assets

del model
model = tf.keras.models.load_model("model_test")

更多详细信息：model 在另一台运行 Linux 的机器（我可以通过我的大学访问的超级计算机）上进行训练，并通过 SCP 传输到我的 Apple M1 机器，它现在显示此加载错误。

我不知道为什么调用 JSON 模块 - 目录中的任何地方似乎都没有 JSON 文件。 但是，鉴于在没有训练和加载的情况下重建 model 没有产生任何错误，我怀疑保存没有正确执行。

Answer 1

在 tensorflow 2.3.1 中遇到了同样的问题。 更新到 2.7，现在又可以工作了！

Answer 2

这是 TensorFlow 版本之间的不兼容性，你训练的 model 和你加载的 model 的 TensorFlow 版本。我的服务器有 TensorFlow 版本 2.6.2，我的电脑有 2.4.1。 在服务器上训练 model 后，当我尝试在我的 PC 上加载时出现“json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)”错误。

• 检查服务器和您的 PC 上的 TensorFlow 版本。 确保它们相似。
• 在我的 PC 上升级 Tensorflow 后，它成功加载了经过训练的 model

Answer 3

你能告诉我你是如何解决这个错误的吗？ 该模型适用于我的 Windows，但在 linux 上出现错误。 这个完全相同的 Json 错误