簡體   English   中英

訓練神經網絡時損失增加

[英]Loss increases when training neural network

我正在嘗試使用神經網絡(用於學習)編寫回歸代碼。

這是我的代碼:

#fixme: k-fold cross validation
n_crossVal = 10
kf = KFold(n_splits = n_crossVal) #, random_state=1, shuffle=True fixme



for p_t in key_set_1:
    
    cur_ds = []
    for i, roi in enumerate(key_set_2):
        if(i==0):
            cur_ds = brain_ds[p_t + '_' + roi]
        else:
            cur_ds = np.hstack((cur_ds, brain_ds[p_t + '_' + roi]))
    
    print(cur_ds.shape)
    print(n_train)
    size_input = cur_ds.shape[1]
    
    preds_case = np.zeros(glove_ds.shape)
    k_no = 0
    for k_train_index, k_test_index in kf.split(cur_ds):
        
        train_X_ds = torch.from_numpy(cur_ds[k_train_index, :])
        train_y_ds = torch.from_numpy(glove_ds[k_train_index, :])
        train_ds   = TensorDataset(train_X_ds, train_y_ds)
        
        test_X_ds  = torch.from_numpy(cur_ds[k_test_index, :])
        test_y_ds = torch.from_numpy(glove_ds[k_test_index, :])
        test_ds   =  TensorDataset(test_X_ds, test_y_ds)
        
        preds = fit_reg(train_ds, train_X_ds, train_y_ds, test_X_ds, test_y_ds, which_case, k_no, p_t)
        k_no += 1
        preds_case[k_test_index, :] = preds.detach().numpy()
        

和我的模型:

class RegressionNet(nn.Module):
    def __init__(self):
        super(RegressionNet, self).__init__()
    
    self.linear1 = nn.Linear(size_input, size_hidden)
    self.act1    = nn.ReLU()
    self.linear2 = nn.Linear(size_hidden, size_output)
    
def forward(self, input_X):
    X = self.linear1(input_X)
    X = self.act1(X)
    X = self.linear2(X)
    return X


def fit_reg(train_ds, train_X_torch, train_y_torch, test_X_torch, case_type, fold_no, p_t):
    num_epochs = 1
    loss_fn = F.mse_loss
    model = RegressionNet()
    opt = torch.optim.SGD(model.parameters(), lr=1e-5)
    
    
    for epoch in range(num_epochs):
        print("num epoch: ", epoch)
        
        for xb, yb in train_ds:
            #not batch? fixme

            
            #print(xb.shape, yb.shape, type(xb), type(yb))
            pred = model(xb.float())
            loss = loss_fn(pred, yb.float())
            
            loss.backward()
            opt.step()
            opt.zero_grad()
            
        print('Training loss: ', loss_fn(model(train_X_torch.float()), train_y_torch.float()))
    
    pred_test_here = model(test_X_torch.float())
    torch.save(model.state_dict(), './weights_' + case_type + '_' +  str(fold_no) + '_' + p_t)
    return pred_test_here

所以我使用 10 折交叉驗證。 每次,我都會將 9/10 的數據傳遞到網絡中,並嘗試在其余數據上進行測試。

我的問題:

  1. 這是執行回歸的正確方法嗎?
  2. 如何發送批量數據而不是每次發送一個樣本進行訓練?
  3. 在訓練完成一定數量的 epoch 后,我將訓練損失顯示為整個樣本之間的損失,對嗎?

提前致謝。

這個問題需要認真編輯; 不包含任何數據,然后使用第二個代碼塊進行數據格式化,這不容易遵循。 但這是我的例子,如果它有幫助的話。 我采用了建議的模型,做了一些小的改動,並制作了一個一維回歸代碼,

import torch
from sklearn.model_selection import KFold
from torch import nn
import math
import torch.nn.functional as F
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
import numpy as np

predictionFull = []
lossFull = []

# data
x = torch.unsqueeze(torch.linspace(-math.pi, math.pi, 1000), dim=1)
y = torch.sin(x**2) + 0.3*torch.rand(x.size())   

fig, ax = plt.subplots(figsize=(12,7))
curve, = ax.plot(x, x, 'r-', linewidth=2)

time_text = ax.text(.5, .5, '', fontsize=15)

def update(i):
    #label = 'timestep {0}'.format(i)
    curve.set_ydata(predictionFull[i].data.numpy())
    time_text.set_text('Loss = %.4f' % lossFull[i].data.numpy())
    time_text.set_x(1.0)
    time_text.set_y(-3.0)
    time_text.set_color('red')

    return curve

class RegressionNet(nn.Module):
    def __init__(self, size_input, size_hidden, size_output):
       super(RegressionNet, self).__init__()
       self.linear1 = nn.Linear(size_input, size_hidden)
       self.linear2 = nn.Linear(size_hidden, size_output)
       
    def forward(self, input_X):
        X = F.relu(self.linear1(input_X)) 
        X = self.linear2(X)
        return X

def fit_reg(x, y):
    num_epochs = 2000
    loss_fn = torch.nn.MSELoss()
    model = RegressionNet(1, 500, 1)
    opt = torch.optim.Adam(model.parameters(), lr=0.002)

    for epoch in range(num_epochs):
            pred = model(x)     # input x and predict based on x
            loss = loss_fn(pred, y)     # must be (1. nn output, 2. target)
            opt.zero_grad()   # clear gradients for next train
            loss.backward()         # backpropagation, compute gradients
            opt.step()        # apply gradients

            predictionFull.append(pred)
            lossFull.append(loss)


fit_reg(x, y)
ax.scatter(x.data.numpy(), y.data.numpy(), color = "orange")
ax.set_xlim(-math.pi, math.pi)
ax.set_ylim(-math.pi, math.pi)

if __name__ == '__main__':
    # FuncAnimation will call the 'update' function for each frame; here
    # animating over 10 frames, with an interval of 200ms between frames.
    anim = FuncAnimation(fig, update, frames=np.arange(0, 2000, 20), interval=2)
    anim.save('./an.gif', writer='imagemagick', fps=500)     

在此處輸入圖片說明

暫無
暫無

聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM