[英]Saving normalization values in Keras model
I have a Keras model for which I would like to save the normalization values in the model
object itself for easier portability.我有一个 Keras 模型,我想将标准化值保存在
model
对象本身中,以便于移植。
I'm using sklearn's StandardScaler()
to normalize my data, so I simply want to save the mean_
and var_
attributes from the scaler
to the model
, save the model, and when I reload the model have access to these attributes.我使用sklearn的
StandardScaler()
规范化我的数据,所以我只是想保存mean_
和var_
从属性scaler
到model
,保存模型,当我重新加载模型都可以访问这些属性。
Currently when I reload the model the attributes I added are not there.目前,当我重新加载模型时,我添加的属性不存在。 What is the correct way of doing this ?
这样做的正确方法是什么?
Code:代码:
# Normalize data
scaler = StandardScaler()
scaler.fit(X_train)
...
# Create model
model = Sequential(...)
# Compile and train
...
# Save model with normalization mean and var
model.normalization_mean = scaler.mean_
model.normalization_var = scaler.var_
keras.models.save_model(model = model,
filepath = ...)
# Reload model
model = keras.models.load_model(filepath = ...)
hasattr(model, 'normalization_mean') # False
hasattr(model, 'normalization_var') # False
this is a possibility... you can create a model subclass in this way and assign external object like not-trainable variables这是一种可能性……您可以通过这种方式创建模型子类并分配外部对象,例如不可训练的变量
X = np.random.uniform(0,1, (100,10))
y = np.random.uniform(0,1, 100)
class MyModel(tf.keras.Model):
def __init__(self):
super(MyModel, self).__init__()
self.dense1 = Dense(32)
self.dense2 = Dense(1)
def call(self, inputs):
x = self.dense1(inputs)
return self.dense2(x)
model = MyModel()
model.compile('adam','mse')
model.fit(X,y)
model._normalization_mean = tf.Variable([111.], trainable=False)
model._normalization_var = tf.Variable([222.], trainable=False)
model.save('abc.tf', save_format='tf')
model = tf.keras.models.load_model(filepath = 'abc.tf')
after loading the model you can call加载模型后,您可以调用
model._normalization_mean.numpy()
# array([111.], dtype=float32)
here the running notebook 这是正在运行的笔记本
to save and load subclass model you can refer to this保存和加载子类模型你可以参考这个
I just came across Keras preprocessing layers whose purpose seem to be exactly what you're describing.我刚刚遇到了Keras 预处理层,其目的似乎正是您所描述的。
The Keras preprocessing layers API allows developers to build Keras-native input processing pipelines.
Keras 预处理层 API 允许开发人员构建 Keras 原生输入处理管道。 These input processing pipelines can be used as independent preprocessing code in non-Keras workflows, combined directly with Keras models, and exported as part of a Keras SavedModel.
这些输入处理管道可以在非 Keras 工作流中用作独立的预处理代码,直接与 Keras 模型结合,并作为 Keras SavedModel 的一部分导出。
With Keras preprocessing layers, you can build and export models that are truly end-to-end: models that accept raw images or raw structured data as input;
使用 Keras 预处理层,您可以构建和导出真正端到端的模型:接受原始图像或原始结构化数据作为输入的模型; models that handle feature normalization or feature value indexing on their own.
自行处理特征归一化或特征值索引的模型。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.