[英]Mixture usage of CPU and GPU in Keras
I am building a neural network on Keras, including multiple layers of LSTM, Permute and Dense. 我正在Keras上构建神经网络,包括LSTM,Permute和Dense的多层。
It seems LSTM is GPU-unfriendly. LSTM似乎对GPU不友好。 So I did research and use
所以我做了研究和使用
With tf.device('/cpu:0'):
out = LSTM(cells)(inp)
But based on my understanding about with
, with
is try...finally
block to ensure that clean-up code is executed. 但是根据我对
with
理解, with
是try...finally
块,以确保执行清理代码。 I don't know whether the following CPU/GPU mixture usage code works or not? 我不知道以下CPU / GPU混合使用代码是否有效? Will they accelerate speed of training?
他们会加快训练速度吗?
With tf.device('/cpu:0'):
out = LSTM(cells)(inp)
With tf.device('/gpu:0'):
out = Permute(some_shape)(out)
With tf.device('/cpu:0'):
out = LSTM(cells)(out)
With tf.device('/gpu:0'):
out = Dense(output_size)(out)
As you may read here - tf.device
is a context manager which switches a default device to this passed as its argument in a context (block) created by it. 您可能会在这里
tf.device
是一个上下文管理器,它将默认设备切换为其在其创建的上下文(块)中作为其参数传递的设备。 So this code should run all '/cpu:0'
device at CPU
and rest on GPU
. 因此,此代码应在
CPU
上运行所有'/cpu:0'
设备,并在GPU
。
The question will it speed up your training is really hard to answer because it depends on the machine you use - but I don't expect computations to be faster as each change of a device makes data to be copied between GPU RAM
and machine RAM
. 能否提高训练速度的问题真的很难回答,因为这取决于您使用的机器-但是我不希望计算速度会更快,因为每次设备更改都会在
GPU RAM
和机器RAM
之间复制数据。 This could even slow down your computations. 这甚至可能减慢您的计算速度。
I have created a model using 2 LSTM and 1 dense layers and trained it in my GPU (NVidia GTX 10150Ti) Here is my observations. 我使用2个LSTM和1个密集层创建了一个模型,并在我的GPU(NVidia GTX 10150Ti)中对其进行了训练。这是我的观察结果。
here is some sample snippet 这是一些示例片段
model = keras.Sequential()
model.add(keras.layers.cudnn_recurrent.CuDNNLSTM(neurons
, batch_input_shape=(nbatch_size, reshapedX.shape[1], reshapedX.shape[2])
, return_sequences=True
, stateful=True))
TojoHere's answer one needs to be upvoted! TojoHere的答案之一需要被投票! This trick made my LSTM training almost 10 times faster.
这个技巧使我的LSTM培训速度提高了近10倍。 Thanks a lot!
非常感谢!
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.