[英]Can a Tensorflow variable be trained using the Tensorflow Keras functional API model? Can a Tensorflow operation be used in the functional API Model?
I am wondering if Keras model compile/training with the functional API train variables defined by tf.get_variable
? 我想知道Keras模型是否使用由
tf.get_variable
定义的功能性API训练变量进行编译/训练? Can Keras training also incorporate Tensorflow operations? Keras培训还可以合并Tensorflow操作吗?
So basically I am looking to define a Keras model with Tensorflow variables and operations, then use 所以基本上我想用Tensorflow变量和操作定义Keras模型,然后使用
model = tf.keras.Model(inputs=inputs, outputs=predictions)
model.compile(optimizer=optimizer, loss=loss)
model.fit(data, labels, batch_size=batch_size, epochs=epochs)
To train the model. 训练模型。 The reason for this is that Google's TPUs require either a Keras or TF.Estimator API, with Keras being more recommended, so I am looking to see how easily I can convert my model.
原因是Google的TPU需要Keras或TF.Estimator API,更推荐使用Keras,因此,我希望了解如何轻松转换模型。
It looks like since Tensorflow is the backend, there are ways to mix Keras/Tensorflow variables. 由于Tensorflow是后端,因此看起来有多种混合Keras / Tensorflow变量的方法。 This blog post shows how Keras variables are trained using a Tensorflow graph/session https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html
这篇博客文章展示了如何使用Tensorflow图/会话https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html训练Keras变量
from keras.layers import Dropout
from keras import backend as K
img = tf.placeholder(tf.float32, shape=(None, 784))
labels = tf.placeholder(tf.float32, shape=(None, 10))
x = Dense(128, activation='relu')(img)
x = Dropout(0.5)(x)
x = Dense(128, activation='relu')(x)
x = Dropout(0.5)(x)
preds = Dense(10, activation='softmax')(x)
loss = tf.reduce_mean(categorical_crossentropy(labels, preds))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(loss)
with sess.as_default():
for i in range(100):
batch = mnist_data.train.next_batch(50)
train_step.run(feed_dict={img: batch[0],
labels: batch[1],
K.learning_phase(): 1})
acc_value = accuracy(labels, preds)
with sess.as_default():
print acc_value.eval(feed_dict={img: mnist_data.test.images,
labels: mnist_data.test.labels,
K.learning_phase(): 0})
And also here it shows that Tensorflow variables can be used as input to a Keras model 而且这里还显示Tensorflow变量可以用作Keras模型的输入
How to set the input of a Keras layer of a functional model, with a Tensorflow tensor? 如何使用Tensorflow张量设置功能模型的Keras层的输入?
tf_embedding_input = ... # pre-processing output tensor
# Keras model
model = Sequential()
model.add(Input(tensor=tf_embedding_input))
model.add(Embedding(max_features, 128, input_length=maxlen))
So I am wondering if Keras can train Tensorflow variables. 所以我想知道Keras是否可以训练Tensorflow变量。
I would like to train the embedding and softmax variables in the Tensorflow architecture below 我想在下面的Tensorflow体系结构中训练embedding和softmax变量
embeddings = tf.get_variable( 'embeddings',
initializer= tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))
softmax_weights = tf.get_variable( 'softmax_weights',
initializer= tf.truncated_normal([vocabulary_size, embedding_size],
stddev=1.0 / math.sqrt(embedding_size)))
softmax_biases = tf.get_variable('softmax_biases',
initializer= tf.zeros([vocabulary_size]), trainable=False )
embed = tf.nn.embedding_lookup(embeddings, train_dataset) #train data set is
embed_reshaped = tf.reshape( embed, [batch_size*num_inputs, embedding_size] )
segments= np.arange(batch_size).repeat(num_inputs)
averaged_embeds = tf.segment_mean(embed_reshaped, segments, name=None)
loss = tf.reduce_mean(
tf.nn.sampled_softmax_loss(weights=softmax_weights, biases=softmax_biases, inputs=averaged_embeds,
labels=train_labels, num_sampled=num_sampled, num_classes=vocabulary_size))
Since Tensorflow Keras uses a Tensorflow backend, I'm guessing it's somehow possible to use and train Tensorflow variables and use Tensorflow operations in training. 由于Tensorflow Keras使用Tensorflow后端,我猜想在某种程度上可以使用和训练Tensorflow变量,并在训练中使用Tensorflow操作。
Google's TPUs require that your architecture be implemented via the Estimator API or Keras API. Google的TPU要求您的架构必须通过Estimator API或Keras API实现。 Since the Keras API is more recommended, there is probably interest in converting a regular Tensorflow Graph/Session to use the Keras API with as few alterations to their code as possible.
由于更推荐使用Keras API,因此可能有兴趣将常规Tensorflow图形/会话转换为使用Keras API,并且对其代码的更改尽可能少。
Knowing how to incorporate Tensorflow operations and train Tensorflow variables using the Keras model compile/train would greatly help with this. 知道如何使用Keras模型编译/训练来合并Tensorflow操作并训练Tensorflow变量将对此有很大帮助。
Little background: 小背景:
As we know Keras is a model-level library, providing high-level building blocks for developing deep learning models. 众所周知,Keras是一个模型级库,为开发深度学习模型提供了高级构建块。
The most important thing: Keras API does not handle tensor operations. 最重要的是:Keras API不处理张量操作。 It needs a well-optimized tensor manipulation library to do so, know as a "backend engine" for Keras.
为此,它需要一个经过优化的张量操纵库,被称为Keras的“后端引擎”。
At this time, Keras has three backend engines available: the TensorFlow backend (Google), the Theano backend, and the CNTK backend (MSFT). 目前,Keras提供了三种后端引擎:TensorFlow后端(Google),Theano后端和CNTK后端(MSFT)。
Knowing how to incorporate Tensorflow operations and train Tensorflow variables using the Keras model compile/train would greatly help with this.
知道如何使用Keras模型编译/训练来合并Tensorflow操作并训练Tensorflow变量将对此有很大帮助。
The only thing you should ask yourself, is what is the difference between the Keras variable and regular Tensorflow variable. 您唯一要问自己的是Keras变量和常规Tensorflow变量之间的区别是什么。
Happens to be that Keras variable have metadata. 可能是Keras变量具有元数据。 So in order to use the TensorFlow variables in Keras you convert them .
因此,为了在Keras中使用TensorFlow变量,需要对其进行转换。
Note: A TensorFlow variable scope will have no effect on a Keras layer or model. 注意:TensorFlow变量作用域对Keras层或模型没有影响。
Finally variable sharing can be done by initializing the Keras layer (or model). 最后,可以通过初始化Keras层(或模型)来完成变量共享。
Would this solution help? 该解决方案有帮助吗?
keras add external trainable variable to graph keras将外部可训练变量添加到图形
You could feed your embeddings and softmax layers into the Keras model using 您可以使用以下方式将嵌入和softmax图层输入Keras模型
model.add()
and then define those variables as trainable using 然后使用定义这些变量为可训练的
model.layers[-1].trainable_weights.extend()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.