来自 tf.nn.conv2d 和 keras.layers.Conv2D 的不等价的 output

Question

我一直在阅读 Aurélien Géron 的动手机器学习教科书（第 2 版）（ 此处为教科书出版商网页）。 我已经进入了将 CNN 应用于图像的内容。 在第 14 章的标题为Tensorflow 实现的部分中，他们手动创建了过滤器，这些过滤器被传递给tf.nn.conv2d并应用于图像以生成一组特征图。 在这些手动过滤器示例之后，这本书说：

在真实的 CNN 中，您通常会将过滤器定义为可训练的变量……而不是手动创建变量，而是使用keras.layers.Conv2D层。

上面的引用对我来说意味着给定相同的输入（和等效的初始化），我们应该能够从tf.nn.conv2d和keras.layers.Conv2D得出相同的输出。 为了验证这个想法，我查看了这两个函数是否等效。 根据这个先前回答的SO帖子，对于卷积，这两个函数是相同的。

我开始对它们的等价性进行一个简单的测试。 我创建了一个卷积层，其中包含一个特征 map 使用 7x7 滤波器（又名：卷积 kernel ） ，所有零分别为tf.nn.conv2d和keras.layers.Conv2D实现正如预期的那样，在对两个图像的差值中的所有像素值求和后，此过滤器确实导致 output 图像的每个像素值的值为零。 这种零差异意味着 output 图像是相同的。

然后我决定创建相同的 7x7 过滤器，但这次全部使用。 理想情况下，两个函数应该产生相同的 output，因此两个 output 图像的差异应该为零。 不幸的是，当我检查 output 图像的差异（并对每个像素的差异求和）时，我得到一个非零和值。 在绘制图像及其差异后，很明显它们不是同一个图像（尽管它们看起来确实非常相似）。

在阅读了这两个函数的文档后，我相信我给了他们等效的输入。 我可以做什么/假设不正确，这会阻止这两个功能产生相同的输出？

我在下面附上了我的代码和版本信息以供参考。 该代码使用 scikit-learn china.jpg示例图像作为输入，并matplotlib.pyplot.imshow来帮助可视化 output 图像及其差异。

TF 版本：2.2.0-dev20200229

Keras 版本：2.3.1

Scikit-Learn 版本：0.22.1

Matplotlib 版本：3.1.3

Numpy 版本：1.18.1

from sklearn.datasets import load_sample_image
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow import keras
import numpy as np

# Get the feature map as a result of tf.nn.conv2d
def featureMap1(batch):
    
    # Extract the channels
    batch_size, height, width, channels = batch.shape

    # Make a (7,7,3,1) filter set (one set of a 7x7 filter per channel)
    # of just ones. 
    filters = np.ones(shape=(7, 7, channels, 1), dtype=np.float32)

    # Run the conv2d with stride of 1 (i.e: in.shape = out.shape)
    # Generate one feature map for this conv layer
    fmaps = tf.nn.conv2d(batch, filters,
                         strides=1, padding='SAME',
                         data_format='NHWC')
    
    # Return the feature map
    return fmaps

# Get the feature map as a result of keras.layers.Conv2D
def featureMap2(batch):

    # Create the input layer with the shape of the images
    inputLayer = keras.layers.Input(shape=batch.shape[1:])
    
    # Create the convLayer which should apply the filter of all ones
    convLayer = keras.layers.Conv2D(filters=1, kernel_size=7,
                                    strides=1, padding='SAME',
                                    kernel_initializer='ones',
                                    data_format='channels_last',
                                    activation='linear')

    # Create the ouput layer
    outputLayer = convLayer(inputLayer)

    # Set up the model
    model = keras.Model(inputs=inputLayer,
                        outputs=outputLayer)

    # Perform a prediction, no model fitting or compiling
    fmaps = model.predict(batch)

    return fmaps 

def main():

    # Get the image and scale the RGB values to [0, 1]
    china = load_sample_image('china.jpg') / 255

    # Build a batch of just one image
    batch = np.array([china])

    # Get the feature maps and extract
    # the images within them
    img1 = featureMap1(batch)[0, :, :, 0]
    img2 = featureMap2(batch)[0, :, :, 0]

    # Calculate the difference in the images
    # Ideally, this should be all zeros...
    diffImage = np.abs(img1 - img2)

    # Add up all the pixels in the diffImage,
    # we expect a value of 0 if the images are
    # identical
    print('Differences value: ', diffImage.sum())

    # Plot the images as a set of 4
    figsize = 10
    f, axarr = plt.subplots(2, 2, figsize=(figsize,figsize))

    axarr[0,0].set_title('Original Image')
    axarr[0,0].imshow(batch[0], cmap='gray')

    axarr[1,0].set_title('Conv2D through tf.nn.conv2d')
    axarr[1,0].imshow(img1, cmap='gray')
    
    axarr[1,1].set_title('Conv2D through keras.layers.Conv2D')
    axarr[1,1].imshow(img2, cmap='gray')

    axarr[0,1].set_title('Diff')
    axarr[0,1].imshow(diffImage, cmap='gray')
    
    plt.show()
    
    return


main()

Answer 1

两个卷积层的 output 应该是相同的。

您将Model与Operation进行比较，而您应该将Operation (tf.keras.Conv2D) 与Operation (tf.nn.conv2d) 进行比较。

修改了featureMap2 function。

def featureMap2(batch):
    # Create the convLayer which should apply the filter of all ones
    convLayer = keras.layers.Conv2D(filters=1, kernel_size = 7,
                                    strides=1, padding='SAME',
                                    kernel_initializer='ones',
                                    data_format='channels_last',
                                    activation='linear')
    fmaps = convLayer(batch)
    return fmaps

这是生成的图。

这是在Google Colab 环境中执行的完整修改代码片段，添加了Seed以确保可重复性并注释掉以前的代码。

%tensorflow_version 2.x

from sklearn.datasets import load_sample_image
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow import keras
import numpy as np

tf.random.set_seed(26)
np.random.seed(26)
tf.keras.backend.set_floatx('float64')


# Get the feature map as a result of tf.nn.conv2d
def featureMap1(batch):

    # Extract the channels
    batch_size, height, width, channels = batch.shape

    # Make a (7,7,3,1) filter set (one set of a 7x7 filter per channel)
    # of just ones. 
    filters = np.ones(shape=(7, 7, channels, 1), dtype=np.float32)

    # Run the conv2d with stride of 1 (i.e: in.shape = out.shape)
    # Generate one feature map for this conv layer
    fmaps = tf.nn.conv2d(batch, filters,
                         strides=1, padding='SAME',
                         data_format='NHWC')

    # Return the feature map
    return fmaps

# Get the feature map as a result of keras.layers.Conv2D
def featureMap2(batch):

    # Create the convLayer which should apply the filter of all ones
    convLayer = keras.layers.Conv2D(filters=1, kernel_size = 7,
                                    strides=1, padding='SAME',
                                    kernel_initializer='ones',
                                    data_format='channels_last',
                                    activation='linear')

    fmaps = convLayer(batch)

    # Create the ouput layer
    # outputLayer = convLayer(inputLayer)

    # # Set up the model
    # model = keras.Model(inputs=inputLayer,
    #                     outputs=outputLayer)

    # Perform a prediction, no model fitting or compiling
    # fmaps = model.predict(batch)

    return fmaps 

def main():

    # Get the image and scale the RGB values to [0, 1]
    china = load_sample_image('china.jpg') / 255

    # Build a batch of just one image
    batch = np.array([china])

    # Get the feature maps and extract
    # the images within them
    img1 = featureMap1(batch)[0, :, :, 0]
    img2 = featureMap2(batch)[0, :, :, 0]
    # Calculate the difference in the images
    # Ideally, this should be all zeros...
    diffImage = np.abs(img1 - img2)

    # Add up all the pixels in the diffImage,
    # we expect a value of 0 if the images are
    # identical
    print('Differences value: ', diffImage.sum())

    # Plot the images as a set of 4
    figsize = 10
    f, axarr = plt.subplots(2, 2, figsize=(figsize,figsize))

    axarr[0,0].set_title('Original Image')
    axarr[0,0].imshow(batch[0], cmap='gray')

    axarr[1,0].set_title('Conv2D through tf.nn.conv2d')
    axarr[1,0].imshow(img1, cmap='gray')

    axarr[1,1].set_title('Conv2D through keras.layers.Conv2D')
    axarr[1,1].imshow(img2, cmap='gray')

    axarr[0,1].set_title('Diff')
    axarr[0,1].imshow(diffImage, cmap='gray')

    plt.show()

    return


main()

编辑：

罪魁祸首是TensorFlow 2.x的默认投射行为。

WARNING:tensorflow:Layer conv2d is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.

由于从float64到float32的精度损失，这会降低计算的准确性。
您可以通过将 Tensorflow Keras 后端默认floatx设置为float64来避免这种精度损失。

tf.keras.backend.set_floatx('float64')