Tensorflow中的自定义渐变操作的grad参数是否始终是一个矩阵？

Question

As far as I am concerned there are two ways of creating custom ops like follows:

@tf.RegisterGradient("CustomGrad")
    def _custom_grad(op, grad):
        return grad

and

@function.Defun(tf.float32, tf.float32)
    def bprop(op, grad):
        return grad

@function.Defun(tf.float32, grad_func=bprop)
    def fprop(W):
        W = tf.sign(W)
        return W

I appears to me that regardless of which type of computation for the forward prop the grad parameter of the custom gradient op is always a matrix of ones. I think it kinda makes sense because you can just use the custom gradient op to let the gradient pass through. However, I need confirmation.

Can someone confirm or correct?

Answer 1

grad is the input gradient from the next instance in the graph after your op with the custom gradient. It is not affected by the forward pass of your op. If you change what happens after the op the grad can change.