[英]Approximating sine function with Neural Network and ReLU
I am trying to approximate a sine function with a neural network (Keras).我正在尝试使用神经网络(Keras)来近似正弦函数。
Yes, I read the related posts :)是的,我阅读了相关帖子:)
Using four hidden neurons with sigmoid and an output layer with linear activation works fine.使用四个带有 sigmoid 的隐藏神经元和一个带有线性激活的输出层工作正常。
But there are also settings that provide results that seem strange to me.但也有一些设置提供的结果对我来说似乎很奇怪。
Since I am just started to work with I am interested in what and why things happen, but I could not figure that out so far.由于我刚刚开始工作,我对事情发生的原因和原因很感兴趣,但到目前为止我无法弄清楚。
# -*- coding: utf-8 -*-
import numpy as np
np.random.seed(7)
from keras.models import Sequential
from keras.layers import Dense
import pylab as pl
from sklearn.preprocessing import MinMaxScaler
X = np.linspace(0.0 , 2.0 * np.pi, 10000).reshape(-1, 1)
Y = np.sin(X)
x_scaler = MinMaxScaler()
#y_scaler = MinMaxScaler(feature_range=(-1.0, 1.0))
y_scaler = MinMaxScaler()
X = x_scaler.fit_transform(X)
Y = y_scaler.fit_transform(Y)
model = Sequential()
model.add(Dense(4, input_dim=X.shape[1], kernel_initializer='uniform', activation='relu'))
# model.add(Dense(4, input_dim=X.shape[1], kernel_initializer='uniform', activation='sigmoid'))
# model.add(Dense(4, input_dim=X.shape[1], kernel_initializer='uniform', activation='tanh'))
model.add(Dense(1, kernel_initializer='uniform', activation='linear'))
model.compile(loss='mse', optimizer='adam', metrics=['mae'])
model.fit(X, Y, epochs=500, batch_size=32, verbose=2)
res = model.predict(X, batch_size=32)
res_rscl = y_scaler.inverse_transform(res)
Y_rscl = y_scaler.inverse_transform(Y)
pl.subplot(211)
pl.plot(res_rscl, label='ann')
pl.plot(Y_rscl, label='train')
pl.xlabel('#')
pl.ylabel('value [arb.]')
pl.legend()
pl.subplot(212)
pl.plot(Y_rscl - res_rscl, label='diff')
pl.legend()
pl.show()
This is the result for four hidden neurons (ReLU) and linear output activation.这是四个隐藏神经元 (ReLU) 和线性输出激活的结果。
Why does the result take the shape of the ReLU?为什么结果采用 ReLU 的形状?
Does this have something to do with the output normalization?这与输出归一化有关吗?
Two things here:这里有两件事:
relu
makes a case when a couple of this neurons are completely saturated highly possible.relu
的神经元很可能会导致其中几个神经元完全饱和。 This is probably why your network result looks like that.he_normal
or he_uniform
as initializer to overcome that.he_normal
或he_uniform
作为初始化程序来克服这个问题。sigmoid
which has a similiar shape to a sin
function this might work fine - but in case of relu
you really need a bigger network.sigmoid
形状与sin
函数相似,这可能会正常工作 - 但在relu
情况下,您确实需要更大的网络。Try adding more hidden layers, each with more hidden units.尝试添加更多隐藏层,每个隐藏层都有更多隐藏单元。 I used this code:
我使用了这个代码:
model = Sequential()
model.add(Dense(50, input_dim=X.shape[1], activation='relu'))
model.add(Dense(50, input_dim=X.shape[1], activation='relu'))
model.add(Dense(1, activation='linear'))
and got these results:并得到了这些结果:
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.