Keras slow on GPU with LSTM Networks

Question

I am training my model with keras. When I compare the performance on GPU vs CPU. The CPU version is much faster as the GPU version

How i can fix these errors below?

I tried to force tensorflow to the GPU, i get these errors:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device to node 'gradients/simple_rnn_1/while/Select_1_grad/Select/f_acc': 
Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available. 
Colocation Debug Info: 
Colocation group had the following types and devices: 
Tile: CPU 
StackPush: GPU CPU 
Relu: GPU CPU 
ReluGrad: GPU CPU 
ZerosLike: GPU CPU 
Select: GPU CPU 
StackPop: GPU CPU 
AddN: GPU CPU 
RefEnter: GPU CPU 
Stack: GPU CPU

My Model looks like this:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
masking_1 (Masking)          (None, None, 3)           0         
_________________________________________________________________
simple_rnn_1 (SimpleRNN)     (None, None, 50)          2700      
_________________________________________________________________
time_distributed_1 (TimeDist (None, None, 11)          561       
_________________________________________________________________
activation_1 (Activation)    (None, None, 11)          0         
=================================================================
Total params: 3,261
Trainable params: 3,261 

EDIT: When i switch the backend to theano, the same net runs much faster on the GPU, i think there is a problem with "tile" on GPU in tensorflow

Answer 1

If you are using ReLU, try use a Tanh loss.

I was using ReLU for a LSTM-DNN model at 100 epochs. Each Epoch went from about 14 second to 2

hopefully this solves your problem - I was running on GPU too so it could just be your processor unfortunately - use google Colab if you are bottlenecked - you get access to free GPU usage