[英]Keras - How to get time taken by each layer in training?
我已經使用 Tensorflow 后端為圖像分類任務實現了 Keras 順序模型。 它有一些自定義層來替換 Keras 層,如 conv2d、max-pooling 等。 但是添加這些層后,雖然保留了准確性,但訓練時間卻增加了數倍。 所以我需要看看這些層是在前向或后向傳遞(通過反向傳播)還是兩者都需要時間,以及這些操作中的哪些可能需要優化(使用 Eigen 等)。
但是,我找不到任何方法來了解模型中每個層/操作所花費的時間。 檢查了 Tensorboard 和回調的功能,但無法了解它們如何幫助計時訓練細節。 有沒有辦法做到這一點? 謝謝你的幫助。
這並不簡單,因為每一層都在每個 epoch 中得到訓練。 您可以使用回調來獲取整個網絡的 epoch 訓練時間,但是您必須進行某種拼湊才能獲得所需的內容(每層的近似訓練時間)。
步驟 -
這不是實際的運行時間,但是,您可以對哪一層比另一層花費的時間按比例增加進行相對分析。
#Callback class for time history (picked up this solution directly from StackOverflow)
class TimeHistory(Callback):
def on_train_begin(self, logs={}):
self.times = []
def on_epoch_begin(self, batch, logs={}):
self.epoch_time_start = time.time()
def on_epoch_end(self, batch, logs={}):
self.times.append(time.time() - self.epoch_time_start)
time_callback = TimeHistory()
# Model definition
inp = Input((inp_dims,))
embed_out = Embedding(vocab_size, 256, input_length=inp_dims)(inp)
x = Conv1D(filters=32, kernel_size=3, activation='relu')(embed_out)
x = MaxPooling1D(pool_size=2)(x)
x = Flatten()(x)
x = Dense(64, activation='relu')(x)
x = Dropout(0.5)(x)
x = Dense(32, activation='relu')(x)
x = Dropout(0.5)(x)
out = Dense(out_dims, activation='softmax')(x)
model = Model(inp, out)
model.summary()
# Function for approximate training time with each layer independently trained
def get_average_layer_train_time(epochs):
#Loop through each layer setting it Trainable and others as non trainable
results = []
for i in range(len(model.layers)):
layer_name = model.layers[i].name #storing name of layer for printing layer
#Setting all layers as non-Trainable
for layer in model.layers:
layer.trainable = False
#Setting ith layers as trainable
model.layers[i].trainable = True
#Compile
model.compile(optimizer='rmsprop', loss='sparse_categorical_crossentropy', metrics=['acc'])
#Fit on a small number of epochs with callback that records time for each epoch
model.fit(X_train_pad, y_train_lbl,
epochs=epochs,
batch_size=128,
validation_split=0.2,
verbose=0,
callbacks = [time_callback])
results.append(np.average(time_callback.times))
#Print average of the time for each layer
print(f"{layer_name}: Approx (avg) train time for {epochs} epochs = ", np.average(time_callback.times))
return results
runtimes = get_average_layer_train_time(5)
plt.plot(runtimes)
#input_2: Approx (avg) train time for 5 epochs = 0.4942781925201416
#embedding_2: Approx (avg) train time for 5 epochs = 0.9014601230621337
#conv1d_2: Approx (avg) train time for 5 epochs = 0.822748851776123
#max_pooling1d_2: Approx (avg) train time for 5 epochs = 0.479401683807373
#flatten_2: Approx (avg) train time for 5 epochs = 0.47864508628845215
#dense_4: Approx (avg) train time for 5 epochs = 0.5149370670318604
#dropout_3: Approx (avg) train time for 5 epochs = 0.48329877853393555
#dense_5: Approx (avg) train time for 5 epochs = 0.4966880321502686
#dropout_4: Approx (avg) train time for 5 epochs = 0.48073616027832033
#dense_6: Approx (avg) train time for 5 epochs = 0.49605698585510255
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.