[英]How can I know if my Neural Network is doing good or not using Mean_Square_Error (Keras)
I'm using Keras and I'm trying to build a Neural Network to predict the interest rate of given data.我正在使用 Keras,我正在尝试构建一个神经网络来预测给定数据的利率。 The data looks like this:
数据如下所示:
loan_amnt annual_inc emp_length int_rate
10000 38000.0 5.600882 12.40
13750 17808.0 5.600882 28.80
26100 68000.0 10.000000 20.00
13000 30000.0 1.000000 20.00
7000 79950.0 7.000000 7.02
The features (X) are loan_amnt
, annual_inc
, and emp_length
.特征 (X) 是
loan_amnt
、 annual_inc
和emp_length
。 The target (y) is int_rate
.目标 (y) 是
int_rate
。
Here's my process and what I've done after normalizing the data:这是我的过程以及我在规范化数据后所做的工作:
#Building out model
model = Sequential([
Dense(9, activation='relu', input_shape=(3,)),
Dense(3, activation='relu'),
Dense(1, activation='linear'),
])
#Compiling model
model.compile(loss='mean_absolute_percentage_error',
metrics=['mse'],
optimizer='RMSprop')
hist = model.fit(X_train, Y_train,
batch_size=100, epochs=20, verbose=1)
Here's an output sample after running model.fit()
:这是运行
model.fit()
后的输出示例:
Epoch 1/20
693/693 [==============================] - 1s 905us/step - loss: 96.2391 - mean_squared_error:
179.8007
Epoch 2/20
693/693 [==============================] - 0s 21us/step - loss: 95.2362 - mean_squared_error:
176.9865
Epoch 3/20
693/693 [==============================] - 0s 20us/step - loss: 94.4133 - mean_squared_error:
174.6367
Finally, evaluating the model model.evaluate(X_train, Y_train)
and got the following output:最后,评估模型
model.evaluate(X_train, Y_train)
并得到以下输出:
693/693 [==============================] - 0s 372us/step
[77.88501817667468, 132.0109032635049]
The question is, how can I know if my model is doing well or not, and how can I read the numbers?问题是,我如何知道我的模型是否运行良好,以及如何读取数字?
You are using a variant of the MSE
loss which is defined as :您正在使用
MSE
损失的一个变体,其定义为:
MSE = mean((y_true - y_pred)^2)
So when you have 132.
as a MSE metrics, then you really have a mean of sqrt(132.)
~= 11,5 mean difference between the y_true and y_pred.所以当你有
132.
作为 MSE 指标时,你真的有一个sqrt(132.)
~= 11,5 y_true 和 y_pred 之间的平均差异。 Which is quite a bit on your data as it is shown on the MSPE
loss, you're having ~78% error on your data.正如
MSPE
损失中显示的那样,这对您的数据有很大影响,您的数据有大约 78% 的错误。
In example if the y_true was 20, you could either predict 36 or 4. Something like that.例如,如果 y_true 是 20,您可以预测 36 或 4。类似的东西。
You could say that your error is good when MSPE is at 10%.当 MSPE 为 10% 时,您可以说您的错误很好。 Depends on your case
取决于你的情况
You should not check the accuracy of your model using the training data because it makes your solution prone to overfitting.您不应使用训练数据检查模型的准确性,因为这会使您的解决方案容易过度拟合。 Instead you should set some data aside (20% is what I usually use) to validate your results.
相反,您应该留出一些数据(我通常使用 20% 的数据)来验证您的结果。
If you plan on doing a lot of testing you should set aside a third dataset only for testing the final solution.如果您计划进行大量测试,则应留出第三个数据集仅用于测试最终解决方案。
You can also use k_folds cross validation where you train the set on part of the data and use the rest to evaluate it, but doing so multiple times to get a better understanding of how accurate your model is.您还可以使用 k_folds 交叉验证,在其中训练部分数据的集合并使用其余部分来评估它,但多次这样做以更好地了解模型的准确度。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.