tf.keras.models.clone_model 未知激活：ReLU

Question

环境是Python 3.7.6，下面是我的导入：

import os, sys
import tensorflow as tf    # v2.2.0
tf.compat.v1.enable_eager_execution()
import numpy as np
import matplotlib.pyplot as plt
from sys import platform
import time
import random
import pickle
from tensorflow.keras.layers import ReLU

我试图克隆一个 tf.keras.Model 没有成功，因为 ReLU 是一个未知的激活。 尽管如此，启动是成功的，所以系统应该知道 ReLU 是什么。 我想知道如何解决这个问题。

def init_model(D=8, W=256):
    # This is a simple MLP neural network
    # D: The number of layers
    # H: The neurons in each layer
    relu = ReLU()
    dense = lambda W=W, act=relu: tf.keras.layers.Dense(W, activation=act, dtype='float32')

    inputs = tf.keras.Input(shape=(3 + 3 * 2 * L_embed))
    outputs = inputs
    for i in range(D):
        outputs = dense()(outputs)
        if i % 4 == 0 and i > 0:
            outputs = tf.concat([outputs, inputs], -1)
    outputs = dense(4, act=None)(outputs)

    model = tf.keras.Model(inputs=inputs, outputs=outputs)
    return model

然后我打电话：

model_inner_copy = tf.keras.models.clone_model(model)

错误信息是：

  File "D:/Code Repository/Meta-NeRF/Code/NeRF/Tiny_MAML_NeRF.py", line 211, in train_maml_nerf
    model_inner_copy = tf.keras.models.clone_model(model)

  File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\models.py", line 427, in clone_model
    model, input_tensors=input_tensors, layer_fn=clone_function)

  File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\models.py", line 196, in _clone_functional_model
    model, new_input_layers, layer_fn)

  File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\models.py", line 246, in _clone_layers_and_model_config
    config = network.get_network_config(model, serialize_layer_fn=_copy_layer)

File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\engine\network.py", line 2119, in get_network_config
    layer_config = serialize_layer_fn(layer)

  File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\models.py", line 243, in _copy_layer
    created_layers[layer.name] = layer_fn(layer)

File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\models.py", line 61, in _clone_layer
    return layer.__class__.from_config(layer.get_config())

File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\engine\base_layer.py", line 655, in from_config
    return cls(**config)

File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\layers\core.py", line 1135, in __init__
    self.activation = activations.get(activation)

File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\activations.py", line 465, in get
    identifier, printable_module_name='activation')

File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\utils\generic_utils.py", line 362, in deserialize_keras_object
    config, module_objects, custom_objects, printable_module_name)

File "C:\Users\Jack\AppData\Roaming\Python\Python37\site-packages\tensorflow\python\keras\utils\generic_utils.py", line 321, in class_and_config_for_serialized_keras_object
    raise ValueError('Unknown ' + printable_module_name + ': ' + class_name)
ValueError: Unknown activation: ReLU

Answer 1

所以问题是tf.keras.layers.ReLU是实现ReLU激活的层，但它本身并不是激活function。 它旨在用作 model 内部的层，而不是用作Dense层的参数。

要有一个 function 作为激活来作为Dense的参数，你应该使用tf.keras.activations.relu 。