TensorFlow CNN tf.nn.conv2d ValueError: Depth of output is not a multiple of the number of groups for (op: 'Conv2D')

Question

I am working on a CNN classification model and try to apply the MNIST data set and get the following ValueError exception during the tf.nn.conv2d() operation, see below for further details.

Code snippet of conv2d operation:

    CONV2_f = 5
    CONV2_n = 16

    ...

    #POOL1

    self.max_pool_layer_1 = tf.nn.max_pool(self.a_convolutional_layer_1, ksize=[ 1, 2, 2, 1 ], strides=[ 1, 2, 2, 1 ], padding='VALID')

    print('POOL1 layer dimension: ', self.max_pool_layer_1.shape)

    #         set_trace()

    #CONV2

    self.filter_convolutional_layer_2 = tf.Variable(tf.random.normal([CONV2_f,CONV2_f, image_channels, CONV2_n]))

    print('self.max_pool_layer_1.shape = ',self.max_pool_layer_1.shape)
    print('self.filter_convolutional_layer_2.shape = ', self.filter_convolutional_layer_2.shape)

    self.conv2d_convolutional_layer_2 = tf.nn.conv2d(self.max_pool_layer_1, use_cudnn_on_gpu=True, filter=self.filter_convolutional_layer_2, strides=[1, 1, 1, 1], padding='VALID')

Print statements:

Input layer dimension:  (1000, 28, 28, 1)
CONV1 layer dimension:  (1000, 24, 24, 6)
POOL1 layer dimension:  (1000, 12, 12, 6)
self.max_pool_layer_1.shape =  (1000, 12, 12, 6)
self.filter_convolutional_layer_2.shape =  (5, 5, 1, 16)

Exception excerpt:

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
C:\dev\programs\Miniconda3\envs\mseml2-gpu\lib\site-packages\tensorflow\python\framework\ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1863   try:
-> 1864     c_op = c_api.TF_FinishOperation(op_desc)
   1865   except errors.InvalidArgumentError as e:

InvalidArgumentError: Depth of output (16) is not a multiple of the number of groups (6) for 'Conv2D_1' (op: 'Conv2D') with input shapes: [1000,12,12,6], [5,5,1,16].

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-21-b36571fb8c02> in <module>
     10     mnist.train_images.shape[3],
     11     batch_size,
---> 12     learning_rate,
     13 )
     14 

<ipython-input-13-d4fafa792491> in __init__(self, num_classes, image_height, image_width, image_channels, batch_size, learning_rate)
     49         print('self.filter_convolutional_layer_2.shape = ', self.filter_convolutional_layer_2.shape)
     50 
---> 51         self.conv2d_convolutional_layer_2 = tf.nn.conv2d(self.max_pool_layer_1, use_cudnn_on_gpu=True, filter=self.filter_convolutional_layer_2, strides=[1, 1, 1, 1], padding='VALID')
     52 
     53         self.biases_convolutional_layer_2 = tf.Variable(tf.random.normal([1, CONV2_n]))

C:\dev\programs\Miniconda3\envs\mseml2-gpu\lib\site-packages\tensorflow\python\ops\nn_ops.py in conv2d(input, filter, strides, padding, use_cudnn_on_gpu, data_format, dilations, name, filters)
   1951                            data_format=data_format,
   1952                            dilations=dilations,
-> 1953                            name=name)
   1954 
   1955 

C:\dev\programs\Miniconda3\envs\mseml2-gpu\lib\site-packages\tensorflow\python\ops\gen_nn_ops.py in conv2d(input, filter, strides, padding, use_cudnn_on_gpu, explicit_paddings, data_format, dilations, name)
   1069                   padding=padding, use_cudnn_on_gpu=use_cudnn_on_gpu,
   1070                   explicit_paddings=explicit_paddings,
-> 1071                   data_format=data_format, dilations=dilations, name=name)
   1072   _result = _op.outputs[:]
   1073   _inputs_flat = _op.inputs

C:\dev\programs\Miniconda3\envs\mseml2-gpu\lib\site-packages\tensorflow\python\framework\op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
    786         op = g.create_op(op_type_name, inputs, dtypes=None, name=scope,
    787                          input_types=input_types, attrs=attr_protos,
--> 788                          op_def=op_def)
    789       return output_structure, op_def.is_stateful, op
    790 

C:\dev\programs\Miniconda3\envs\mseml2-gpu\lib\site-packages\tensorflow\python\util\deprecation.py in new_func(*args, **kwargs)
    505                 'in a future version' if date is None else ('after %s' % date),
    506                 instructions)
--> 507       return func(*args, **kwargs)
    508 
    509     doc = _add_deprecated_arg_notice_to_docstring(

C:\dev\programs\Miniconda3\envs\mseml2-gpu\lib\site-packages\tensorflow\python\framework\ops.py in create_op(***failed resolving arguments***)
   3614           input_types=input_types,
   3615           original_op=self._default_original_op,
-> 3616           op_def=op_def)
   3617       self._create_op_helper(ret, compute_device=compute_device)
   3618     return ret

C:\dev\programs\Miniconda3\envs\mseml2-gpu\lib\site-packages\tensorflow\python\framework\ops.py in __init__(self, node_def, g, inputs, output_types, control_inputs, input_types, original_op, op_def)
   2025           op_def, inputs, node_def.attr)
   2026       self._c_op = _create_c_op(self._graph, node_def, grouped_inputs,
-> 2027                                 control_input_ops)
   2028 
   2029     # Initialize self._outputs.

C:\dev\programs\Miniconda3\envs\mseml2-gpu\lib\site-packages\tensorflow\python\framework\ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1865   except errors.InvalidArgumentError as e:
   1866     # Convert to ValueError for backwards compatibility.
-> 1867     raise ValueError(str(e))
   1868 
   1869   return c_op

ValueError: Depth of output (16) is not a multiple of the number of groups (6) for 'Conv2D_1' (op: 'Conv2D') with input shapes: [1000,12,12,6], [5,5,1,16].

The main issue seems to be: ValueError: Depth of output (16) is not a multiple of the number of groups (6) for 'Conv2D_1' (op: 'Conv2D') with input shapes: [1000,12,12,6], [5,5,1,16].

The fun fact is that I already used my CNN classification model for classification of cat data and it worked with same depth of output and number of groups.

Hardfacts:

• Windows 10
• Anaconda Powershell for Windows (Version 4.7.12)
• Python (Version 3.7.4.final.0)
• Using GPU acceleration
• CUDU (Version 10.0)
• cuDNN (Version 7.4.2)
• Environment dependencies (environment.yml):

name: PLACEHOLDER
channels:
 - defaults
dependencies:
 - jupyter==1.0.0
 - matplotlib==3.1.1
 - numpy==1.16.5
 - scikit-learn==0.21.2
 - scipy==1.3.1
 - tensorboard==1.14.0
 - tensorflow-gpu==1.14.0
 - python==3.7.0

Have you any idea what I am doing wrong?

I thank you for answers and tips in advance!

Answer 1

I found the solution some time later, thanks to a friend.

Instead of:

    self.filter_convolutional_layer_2 = tf.Variable(tf.random.normal([CONV2_f,CONV2_f, image_channels, CONV2_n]))

Use:

    self.filter_convolutional_layer_2 = tf.Variable(tf.random.normal([CONV2_f,CONV2_f, self.max_pool_layer_1.shape[3], CONV2_n]))