張量流中的權重和偏差初始化

Question

我正在做一些電力負荷預測，在其中我要初始化權重和偏差。 我已經使用不同的算法計算了體重和偏倚，並將其保存在文件中。 我想使用該文件，並使用這些權重和偏見開始我的訓練。

這是我要更新的代碼。

#RNN designning
tf.reset_default_graph()

inputs = 1  #input vector size
hidden = 100    
output = 1  #output vector size

X = tf.placeholder(tf.float32, [None, num_periods, inputs])
y = tf.placeholder(tf.float32, [None, num_periods, output])


basic_cell = tf.contrib.rnn.BasicRNNCell(num_units=hidden, activation=tf.nn.relu)
rnn_output, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32)

learning_rate = 0.001   #small learning rate so we don't overshoot the minimum

stacked_rnn_output = tf.reshape(rnn_output, [-1, hidden])           #change the form into a tensor
stacked_outputs = tf.layers.dense(stacked_rnn_output, output)        #specify the type of layer (dense)
outputs = tf.reshape(stacked_outputs, [-1, num_periods, output])          #shape of results

loss = tf.reduce_mean(tf.square(outputs - y))    #define the cost function which evaluates the quality of our model
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)          #gradient descent method
training_op = optimizer.minimize(loss)          #train the result of the application of the cost_function                                 

init = tf.global_variables_initializer()           #initialize all the variables
epochs = 1000     #number of iterations or training cycles, includes both the FeedFoward and Backpropogation
mape = []

def mean_absolute_percentage_error(y_true, y_pred): 
    y_true, y_pred = np.array(y_true), np.array(y_pred)
    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100

y_pred = {'NSW': [], 'QLD': [], 'SA': [], 'TAS': [], 'VIC': []}

for st in state.values():
    print("State: ", st, end='\n')
    with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as sess:
        init.run()
        for ep in range(epochs):
            sess.run(training_op, feed_dict={X: x_batches[st], y: y_batches[st]})
            if ep % 100 == 0:
                mse = loss.eval(feed_dict={X: x_batches[st], y: y_batches[st]})
                print(ep, "MSE:", mse)
        y_pred[st] = sess.run(outputs, feed_dict={X: x_batches_test[st]})
    print("\n")

我正在使用以下算法找到權重和偏差，並將其保存在weights和biases列表中。

class network:
    def set_weight_bias(self, a):
        lIt = 0
        rIt = 0
        self.weights = []
        self.biases = []
        for x,y in zip(self.sizes[1:], self.sizes[:-1]):
            rIt += x*y
            self.weights.append(a[lIt:rIt].reshape((x,y)))
            lIt = rIt
        for x in self.sizes[1:]:
            rIt += x
            self.biases.append(a[lIt:rIt].reshape((x,1)))
            lIt = rIt

    ...
    """
    Cuckoo Search Optimization
    """

    def objectiveFunction(self,x):
        self.set_weight_bias(x)
        y_prime = self.feedforward(self.input)
        return sum(abs(u-v) for u,v in zip(y_prime, self.output))/x.shape[0]

    def cso(self, n, x, y, function, lb, ub, dimension, iteration, pa=0.25,
                 nest=100):
        """
        :param n: number of agents
        :param function: test function
        :param lb: lower limits for plot axes
        :param ub: upper limits for plot axes
        :param dimension: space dimension
        :param iteration: number of iterations
        :param pa: probability of cuckoo's egg detection (default value is 0.25)
        :param nest: number of nests (default value is 100)
        """
        ...

我想使用自定義權重和偏差開始訓練，而不是通過tensorflow隨機分配權重和偏差。 如何在張量流中做到這一點？

Answer 1

對於每一層，您可以參考文檔以了解如何完成初始化：

• 在set_weights上調用set_weights函數（ docs ）
• 傳遞一個將初始權重返回給kernel_initializer ，並將一個將初始偏差返回給bias_initializer同時創建密集層（ docs ）

Answer 2

您是否要為RNN單元或密集層設置權重？ 如果用於RNN單元，則應該能夠使用set_weights方法設置權重。

如果用於密集層，則應該能夠分配一個Variable並使用initializer參數傳遞權重（另一個用於bias'）。 然后，當您調用layers.dense ，可以將變量張量分別傳遞給kernel_initializer和bias_initializer以獲得權重和偏差。