ValueError：尺寸必须相等，但对于具有输入形状的“p_softmax/truediv”（操作：“RealDiv”）为 3 和 300：[?,300,300,3]、[?,300,300]

Question

I am currently trying to create a parametric version of Softmax, and I receive the above error, when making my axis hyperparam -1 for 3D space (color+x+y). I'd like to know whether my implementation is correct for 3D space, and if it is only correct for 2d or 1d space, then how can I generalize Softmax for 3d spaces?

Code for layer and testing:

import random
import numpy as np
from tensorflow import keras
import tensorflow as tf
from tensorflow.keras import layers as L

class PSoftmax(L.Layer):
    def __init__(self, axis=-1):
        super(PSoftmax, self).__init__()

        self.weight = self.add_weight(shape=(1,1),
                                    initializer='random_normal'
                                    )
        self.axis = axis
    def call(self, inputs):
        y = tf.pow(tf.abs(self.weight), inputs)
        return y / tf.reduce_sum(y, axis=self.axis)

y = L.Input(shape=(300,300,3))
x = PSoftmax(-1)(y)

Traceback:

Traceback (most recent call last):
  File "layers.py", line 22, in <module>
    y = PSoftmax()(y)
  File "/home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 842, in __call__
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "/home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/autograph/impl/api.py", line 237, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in converted code:

    layers.py:19 call  *
        return y / tf.reduce_sum(y, axis=self.axis)
    /home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/math_ops.py:899 binary_op_wrapper
        return func(x, y, name=name)
    /home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/math_ops.py:1005 _truediv_python3
        return gen_math_ops.real_div(x, y, name=name)
    /home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_math_ops.py:7954 real_div
        "RealDiv", x=x, y=y, name=name)
    /home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py:793 _apply_op_helper
        op_def=op_def)
    /home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py:548 create_op
        compute_device)
    /home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:3429 _create_op_internal
        op_def=op_def)
    /home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1773 __init__
        control_input_ops)
    /home/ai/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1613 _create_c_op
        raise ValueError(str(e))

    ValueError: Dimensions must be equal, but are 3 and 300 for 'p_softmax/truediv' (op: 'RealDiv') with input shapes: [?,300,300,3], [?,300,300].

Answer 1

You have to keep the dimension, when you call tf.reduce_sum :

def call(self, inputs):
    y = tf.pow(tf.abs(self.weight), inputs)
    print(y)
    return y / tf.reduce_sum(y, axis=self.axis, keepdims=True)

Otherwise, the last dimension (channel) is reduced and you try to divide a tensor of shape [None, 300 , 300] with a tensor of shape [None, 300, 300, 3] as the error states.