PyTorch 线性回归模型

Question

I have a multivariate linear regression problem in which each data point looks like this:

y_i = 3                             # Some integer between 0 and 20
X_i = [0.5, 80, 0.004, 0.5, 0.789]  # A 5 dimensional vector

I can train a simple linear model by using sklearn, something like:

from sklearn import linear_model
ols = linear_model.LinearRegression()
model = ols.fit(X, y)

This gets me an accuracy of ~55 % (a linear model is not suitable for the problem, but this is a baseline to demonstrate the feasibility of modelling the problem, and a way for me to learn PyTorch, having use TensorFlow previously).

When I try to train a linear model using PyTorch I am defining the model as:

class TwoLayerNet(torch.nn.Module):
    def __init__(self, D_in, D_out):

        super(TwoLayerNet, self).__init__()
        self.linear1 = torch.nn.Linear(D_in, D_out)

    def forward(self, x):

        y_pred = self.linear1(x)
        return y_pred

D_in, D_out = 5, 1
model = TwoLayerNet(D_in, D_out)

And training as:

epochs = 10
criterion = torch.nn.MSELoss(reduction='sum')
optimizer = torch.optim.SGD(model.parameters(), lr=1e-4)
for epoch in range(epochs):
    for n, batch in enumerate(batches):
        X = []
        y = []
        for values in batch:
            X.append(values[0])
            y.append(values[1])
        
        X = torch.from_numpy(np.asarray(X))
        y = torch.from_numpy(np.asarray(y))
        # Forward pass: Compute predicted y by passing x to the model
        optimizer.zero_grad()
        y_pred = model(X)
        # Compute and print loss
        loss = criterion(y_pred, y)
        if n % 100 == 99:
            print(n, loss.item())

        # Zero gradients, perform a backward pass, and update the weights.
        
        loss.backward()
        optimizer.step()

This is just some code from the PyTorch documentation which I have adjusted. The current set up only achieves ~25%, no where near the accuracy that I would expect from the linear model. Am I doing something incorrect in the model training wrt PyTorch?

Answer 1

tam63,

you are missing activation function in the model definition. replace

y_pred = self.linear1(x)

with

y_pred =  F.relu(self.linear1(x))

there are few more things that may go wrong. For instance (1) too low a learning rate, (2) too few layers (add one more). If you are familiar with TF as you say, try same problem in TF and once you have good results - translate it into Pytorch with same network structure and same hyperparameters.