Pytorch 归一化二维张量

Question

For more robustnes of my model I want to normalize my feature tensor.

I tried doing it the way that is to the best of my knowledge standard for pictures:

class Dataset(torch.utils.data.Dataset):
'Characterizes a dataset for PyTorch'
def __init__(self, input_tensor, transform = transforms.Normalize(mean= 0.5, std=0.5)):
    self.labels = input_tensor[:,:,-1]
    self.features = input_tensor[:,:,:-1]
    self.transform = transform

def __len__(self):
    return self.labels_planned.shape[0]

def __getitem__(self, index):

    # Load data and get label
    X = self.features[index]
    y = self.labelslabels[index]
    if self.transform:
        X = self.transform(X)   
    return X, y

But receive this error message:

ValueError: Expected tensor to be a tensor image of size (C, H, W). Got tensor.size() = torch.Size([8, 25]).

Everywhere I looked people suggest that one should use.view to generate the third dimension in order to comply with the standard shape of pictures, but this seems very odd to me. Is there maybe a cleaner way to do this. Also where should I best place the normalization? Just for the batch or for the entire train dataset?

Answer 1

You are asking two different questions, I will try to answer both.

• Indeed, you should first reshape to (c, h, w) where c is the channel dimension In most cases, you will need that extra dimension because most 'image' layers are built to receive 3d dimensional tensors - not counting the batch dimension - such as nn.Conv2d , BatchNorm2d , etc... I don't believe there's anyways around it, and doing so would restrict yourself to one-layer image datasets.

  You can broadcast to the desired shape with torch.reshape or Tensor.view :

   X = X.reshape(1, *X.shape)

  Or by adding an additional dimension using torch.unsqueeeze :

   X.unsqueeze(0)

• About normalization. Batch-normalization and dataset-normalization are two different approaches.

  The former is a technique that can achieve improved performance in convolution networks. This kind of operation can be implemented using a nn.BatchNorm2d layer and is done using learnable parameters: a scale factor (~ std) and a bias (~ mean). This type of normalization is applied when the model is called and is applied per-batch.

  The latter is a pre-processing technique which allows making different features have the same scale. This normalization can be applied inside the dataset per-element. It requires you measure the mean and standard deviation of your training set.