如何使用Keras + Tensorflow测量重复试验的解决时间

Question

Tensorflow培训的启动成本可通过重复试验进行摊销。 我想在没有这个启动成本的情况下测量训练模型问题（XOR）的时间，我想要进行多次试验。 每个试验都需要从新的初始条件开始，否则它只是重复找到相同的解决方案。

我想我需要重置每个试验的权重，并使用EarlyStopping回调，以便在无法再改进解决方案时停止培训。 然后我报告经过的时间和损失。

import keras
import numpy
import math
import sys
from keras.initializers import glorot_uniform
import keras.backend as K

if len(sys.argv) < 2:
  print('tell me how many samples to generate')
  sys.exit(-1)

numSamples = int(sys.argv[1])


learningRate = 0.1

model = keras.models.Sequential()
model.add(keras.layers.Dense(2, activation='sigmoid', input_dim=2))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.compile(loss=keras.losses.binary_crossentropy, optimizer=keras.optimizers.Adam(lr=learningRate), metrics=['accuracy'])

initial_weights = model.get_weights()

backend_name = K.backend()
if backend_name == 'tensorflow': 
    k_eval = lambda placeholder: placeholder.eval(session=K.get_session())
elif backend_name == 'theano': 
    k_eval = lambda placeholder: placeholder.eval()
else: 
    raise ValueError("Unsupported backend")


X = [ [ 0, 0 ], [ 0, 1 ], [ 1, 0 ], [ 1, 1 ] ]
Y = [ [ 1 ], [ 0 ], [ 0 ], [ 1 ] ]

XX = numpy.array(X)
YY = numpy.array(Y)

lastModel = None

earlyStopping = keras.callbacks.EarlyStopping(monitor='loss', min_delta=0, patience=10, verbose=1)

from timeit import default_timer as timer

for i in range(numSamples):
  new_weights = [k_eval(glorot_uniform()(w.shape)) for w in initial_weights]
  model.set_weights(new_weights)

  start = timer()

  model.fit(XX, YY, verbose=0, epochs=1000000, callbacks=[earlyStopping], validation_data=(XX, YY))

  end = timer()
  elapsed = end - start
  print("Elapsed seconds to fit new solution:", elapsed)

  score = model.evaluate(XX, YY, verbose=0)
  print()
  print('loss', score)

  layer0 = model.layers[0].get_weights()
  layer1 = model.layers[1].get_weights()

  weight0 = layer0[0]
  bias0 = layer0[1]
  w1 = weight0[0][0]
  w2 = weight0[0][1]
  w3 = weight0[1][0]
  w4 = weight0[1][1]
  b1 = bias0[0]
  b2 = bias0[1]

  weight1 = layer1[0]
  bias1 = layer1[1]
  w5 = weight1[0][0]
  w6 = weight1[1][0]
  b3 = bias1[0]

  w = [w1, w2, w3, w4, w5, w6, b1, b2, b3]
  print(w)

  if lastModel is not None:
    sum = 0
    for i in range(len(w)):
      sum = sum + (w[i] - lastModel[i]) * (w[i] - lastModel[i])
    rms = math.sqrt(sum) / len(w)
    print("rms from last solution = ", rms)
  lastModel = w
  for i in range(len(X)):
    z01 = X[i][0] * w1 + X[i][1] * w3 + b1
    z11 = X[i][0] * w2 + X[i][1] * w4 + b2
    a01 = 1.0 / (1.0 + math.exp(-z01))
    a11 = 1.0 / (1.0 + math.exp(-z11))

    z2 = a01 * w5 + a11 * w6 + b3
    yHat = 1.0 / (1.0 + math.exp(-z2))
    print(X[i], Y[i], yHat)

令我感到惊讶的是，有些时间是在几秒钟内，而其他时间是在几分之一秒内。 我不希望XOR要求多秒训练。 有没有人发现这个程序或方法有什么问题？