Keras中的IDct自定义图层

Question

I am trying to write a custom layer in Keras for IDCT (Inverse Discrete Cosine Transform) as there is no built-in function in Keras for IDCT as compared to DCT. So when I write my layer as:

model = Sequential()
model.add(Conv2D(512,1,activation='relu', input_shape= (8,8,64) ))
model.add(Lambda( lambda x: get_2d_idct_tensor(x) ) )

where my function is defined as :

def get_2d_idct_tensor(coefficients):
   return fftpack.idct(K.transpose(fftpack.idct(K.transpose(coefficients), norm='ortho')), norm='ortho')

I get the following error:

----> 9 model.add(Lambda( lambda x: get_2d_idct_tensor(x) ) )
 10 
 11 #model.add(Lambda(lambda x: K.tf.spectral.dct(K.transpose(K.tf.spectral.dct(K.transpose(x), type=2, norm='ortho')), norm='ortho'),input_shape=(8, 8, 512),output_shape=(8, 8, 1) ))

/usr/local/lib/python3.6/dist-packages/keras/models.py in add(self, layer)
520                           output_shapes=[self.outputs[0]._keras_shape])
521         else:
--> 522             output_tensor = layer(self.outputs[0])
523             if isinstance(output_tensor, list):
524                 raise TypeError('All layers in a Sequential model '

/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py in __call__(self, inputs, **kwargs)
617 
618             # Actually call the layer, collecting output(s), mask(s), and shape(s).
--> 619             output = self.call(inputs, **kwargs)
620             output_mask = self.compute_mask(inputs, previous_mask)
621 

/usr/local/lib/python3.6/dist-packages/keras/layers/core.py in call(self, inputs, mask)
683         if has_arg(self.function, 'mask'):
684             arguments['mask'] = mask
--> 685         return self.function(inputs, **arguments)
686 
687     def compute_mask(self, inputs, mask=None):

<ipython-input-14-dae1f7021aae> in <lambda>(x)
  7 model.add(Conv2D(512,1,activation='relu', input_shape= (8,8,64) ))
  8 
----> 9 model.add(Lambda( lambda x: get_2d_idct_tensor(x) ) )
 10 
 11 #model.add(Lambda(lambda x: K.tf.spectral.dct(K.transpose(K.tf.spectral.dct(K.transpose(x), type=2, norm='ortho')), norm='ortho'),input_shape=(8, 8, 512),output_shape=(8, 8, 1) ))

<ipython-input-7-9ac404754077> in get_2d_idct_tensor(coefficients)
 12     """ Get 2D Inverse Cosine Transform of Image
 13     """
---> 14     return fftpack.idct(K.transpose(fftpack.idct(K.transpose(coefficients), norm='ortho')), norm='ortho')
 15 
 16 def get_reconstructed_image(img):

/usr/local/lib/python3.6/dist-packages/scipy/fftpack/realtransforms.py in idct(x, type, n, axis, norm, overwrite_x)
200     # Inverse/forward type table
201     _TP = {1:1, 2:3, 3:2}
--> 202     return _dct(x, _TP[type], n, axis, normalize=norm, overwrite_x=overwrite_x)
203 
204 

/usr/local/lib/python3.6/dist-packages/scipy/fftpack/realtransforms.py in _dct(x, type, n, axis, overwrite_x, normalize)
279 
280     """
--> 281     x0, n, copy_made = __fix_shape(x, n, axis, 'DCT')
282     if type == 1 and n < 2:
283         raise ValueError("DCT-I is not defined for size < 2")

/usr/local/lib/python3.6/dist-packages/scipy/fftpack/realtransforms.py in __fix_shape(x, n, axis, dct_or_dst)
224 
225 def __fix_shape(x, n, axis, dct_or_dst):
--> 226     tmp = _asfarray(x)
227     copy_made = _datacopied(tmp, x)
228     if n is None:

/usr/local/lib/python3.6/dist-packages/scipy/fftpack/basic.py in _asfarray(x)
125     already an array with a float dtype, and do not cast complex types to
126     real."""
--> 127     if hasattr(x, "dtype") and x.dtype.char in numpy.typecodes["AllFloat"]:
128         # 'dtype' attribute does not ensure that the
129         # object is an ndarray (e.g. Series class

AttributeError: 'DType' object has no attribute 'char'

Can someone please explain what is the error trying to say and why is it caused? I am pretty new to Keras and would like some help to point me in the right direction.

Thanks in advance for your time and help...

Answer 1

You are running an operation which expects a NumPy ndarray on tensors. Unfortunately this will not work. You need to reimplement the custom operation using only tensor operators.

Having said that, using functions from Tensorflow directly is OK as well, say from import tensorflow and use those inside a custom layer might give you more functions than Keras backend alone.