EfficientNetB3 架構制作 Class 激活 Map (CAM)

Question

我想為基於 EfficeintNet B3 的 model 繪制一個 class 激活 map。 但是當我遵循不同來源的不同教程和代碼時，它就失敗了……

#load images
img = tf.keras.preprocessing.image.load_img(
        base, target_size=(img_height, img_width))
img_array = tf.keras.preprocessing.image.img_to_array(img)
img_array = tf.expand_dims(img_array, 0) # Create a batch
predictions = model.predict(img_array)
score = tf.nn.softmax(predictions[0])


last_conv = model.layers[2].layers[-3]

grad_model = tf.keras.models.Model(
    [model.inputs], [last_conv.output, model.output])

無法構建 grad_model

ValueError：圖形斷開連接：無法獲取張量 KerasTensor(type_spec=TensorSpec(shape=(None, 300, 300, 3), dtype=tf.float32, name='input_1'), name='input_1', description="由“input_1”層“stem_conv”層創建。 可以毫無問題地訪問以下先前的層：[]

這是 model：

 Model: "sequential_1"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #   
    =================================================================
    sequential (Sequential)      (None, 300, 300, 3)       0         
    _________________________________________________________________
    rescaling (Rescaling)        (None, 300, 300, 3)       0         
    _________________________________________________________________
    efficientnet-b3 (Functional) (None, 10, 10, 1536)      10783528  
    _________________________________________________________________
    global_average_pooling2d (Gl (None, 1536)              0         
    _________________________________________________________________
    dropout (Dropout)            (None, 1536)              0         
    _________________________________________________________________
    dense (Dense)                (None, 128)               196736    
    _________________________________________________________________
    dense_1 (Dense)              (None, 5)                 645       
    =================================================================

Answer 1

要解決圖形斷開值錯誤，您需要正確構建 grad cam model。 這是為 grad-cam 構建 model 的方法之一。

import tensorflow as tf 
from tensorflow import keras
from tensorflow.keras import layers

inputs = tf.keras.Input(shape=(300, 300, 3))

x = keras.applications.EfficientNetB3(
    input_tensor=inputs, # pass input to input_tensor 
    include_top=False,
    weights=None
)

# flat the base model with x.output 
x = layers.GlobalAveragePooling2D()(x.output) 

# others 
x = layers.Dense(128)(x)
x = layers.Dropout(0.5)(x)
x = layers.Dense(5)(x)

model = keras.Model(inputs, x)

for i, layer in enumerate(model.layers[-10:]):
    print(i, layer.name, layer.output_shape, layer.trainable)

0 block7b_project_bn (None, 10, 10, 384) True
1 block7b_drop (None, 10, 10, 384) True
2 block7b_add (None, 10, 10, 384) True
3 top_conv (None, 10, 10, 1536) True
4 top_bn (None, 10, 10, 1536) True
5 top_activation (None, 10, 10, 1536) True  # < - We will pick this 2D maps
6 global_average_pooling2d_2 (None, 1536) True
7 dense_2 (None, 128) True
8 dropout (None, 128) True
9 dense_3 (None, 5) True

構建 Grad-CAM Model

grad_model = keras.models.Model(
    [model.inputs], 
    [
        model.get_layer('top_activation').output, 
        model.output
     ]
)

查看

使用您的設置，您會收到以下代碼的斷開連接錯誤。 但現在，這不會發生。

import numpy as np 

image = np.random.rand(1, 300, 300, 3).astype(np.float32) 

with tf.GradientTape() as tape:
    convOutputs, predictions = grad_model(tf.cast(image, tf.float32))
    loss = predictions[:, tf.argmax(predictions[0])]

grads = tape.gradient(loss, convOutputs)
print(grads.shape) 
(1, 10, 10, 1536) # NO DISCONNECTED ERROR

要從您的 grad-cam model 獲取熱圖，請檢查以下答案和來源作為參考。

• 研究生CAM
• 引導式 GradCAM
• Grad-CAM 多輸出 Model
• GradCAM-回調
• 雙變形金剛：GradCAM