[英]Implementing same convolution in Pytorch
I was trying to implement same convolution.我试图实现相同的卷积。 It seems (surprisingly) that such a standard layer isn't available out of the box in Pytorch.似乎(令人惊讶的是)这样的标准层在 Pytorch 中无法开箱即用。 I hope this helps make sure a layer available.我希望这有助于确保图层可用。 I was needing help generalizing it to use ALL hyper parameters.我需要帮助概括它以使用所有超参数。 Right now it only uses kernel size (ie variable stride, dialation or whatever else that might affect things isn't part of this. Anyone know how to generalize it with all hyper params of a Convolution (in Conv Nets)?现在它只使用内核大小(即可变步长、拨号或任何其他可能影响事物的东西不是这个的一部分。有人知道如何用卷积的所有超参数来概括它(在卷积网络中)?
Code:代码:
def get_cnn(C, out_filters=[14,13,12,3], kernels=[(9,7),(7,7),(5,3),(3,3)]):
N = len(out_filters)
print()
print(f'C = {C}')
model_layers = OrderedDict()
in_channels = C
x = torch.randn(1,C,10,10)
out = x
print(f'initial_size = {x.size()}')
for i in range(1,N):
print(f'--- building layer = {i}')
## create conv layer
kernel_size = kernels[i]
out_channels = out_filters[i]
# to make sure its a same convolution (ignoring stride)
print(f'in_channels = {in_channels}')
print(f'out_channels = {out_channels}')
padding = ((kernels[i][0]-1)//2,(kernels[i][1]-1)//2)
conv = torch.nn.Conv2d(in_channels, out_channels, kernel_size, padding=padding)
#conv = torch.nn.Conv2d(in_channels, out_channels, kernel_size)
model_layers[f'Conv{i}'] = conv
## compute dummy data
out = conv(out)
print(f'x.size() = {x.size()}')
print(f'out.size() = {out.size()}')
## add activation
model_layers[f'ReLU{i}'] = torch.nn.ReLU()
## so that next layer works
in_channels = out_channels
## make sequential model
mdl = torch.nn.Sequential(model_layers)
y = mdl(x)
return mdl
References:参考:
Does padding ensure that the sizes of input and output of a CNN match?填充是否确保 CNN 的输入和输出大小匹配? : https://stats.stackexchange.com/questions/419902/does-padding-ensure-that-the-sizes-of-input-and-output-of-a-cnn-match : https : //stats.stackexchange.com/questions/419902/does-padding-ensure-that-the-sizes-of-input-and-output-of-a-cnn-match
arithmetic of convolutions: https://arxiv.org/abs/1603.07285卷积算法: https : //arxiv.org/abs/1603.07285
https://discuss.pytorch.org/t/same-convolution-in-pytorch/19937 https://discuss.pytorch.org/t/same-convolution-in-pytorch/19937
https://discuss.pytorch.org/t/how-do-i-make-sure-the-output-of-a-cnn-never-decrease-in-size-using-padding/52103 https://discuss.pytorch.org/t/how-do-i-make-sure-the-output-of-a-cnn-never-decrease-in-size-using-padding/52103
Full running code:完整运行代码:
import torch
from collections import OrderedDict
import random
def do_conv():
x = torch.randn(1,3,5,6)
conv = torch.nn.Conv2d(in_channels=3, out_channels=5, kernel_size=(3,3))
y = conv(x)
print(y)
def get_cnn(C, out_filters=[14,13,12,3], kernels=[(9,7),(7,7),(5,3),(3,3)]):
N = len(out_filters)
print()
print(f'C = {C}')
model_layers = OrderedDict()
in_channels = C
x = torch.randn(1,C,10,10)
out = x
print(f'initial_size = {x.size()}')
for i in range(1,N):
print(f'--- building layer = {i}')
## create conv layer
kernel_size = kernels[i]
out_channels = out_filters[i]
# to make sure its a same convolution (ignoring stride)
print(f'in_channels = {in_channels}')
print(f'out_channels = {out_channels}')
padding = ((kernels[i][0]-1)//2,(kernels[i][1]-1)//2)
conv = torch.nn.Conv2d(in_channels, out_channels, kernel_size, padding=padding)
#conv = torch.nn.Conv2d(in_channels, out_channels, kernel_size)
model_layers[f'Conv{i}'] = conv
## compute dummy data
out = conv(out)
print(f'x.size() = {x.size()}')
print(f'out.size() = {out.size()}')
## add activation
model_layers[f'ReLU{i}'] = torch.nn.ReLU()
## so that next layer works
in_channels = out_channels
## make sequential model
mdl = torch.nn.Sequential(model_layers)
y = mdl(x)
return mdl
def get_hardcoded_variable_size_input():
C = 3
variable_size_input = [torch.randn(1,C,15,15), torch.randn(1,C,12,16), torch.randn(1,C,15,11)]
return variable_size_input
def get_variable_size_input(N=3):
variable_size_input = []
for i in range(N):
C,H,W = 3, random.randint(11,20), random.randint(14,19)
x = torch.randn(C,H,W)
variable_size_input.append(x)
return variable_size_input
def any_input_output_equals_input():
'''
test that no matter what input, the output of the network is same as the input
'''
X_list = get_hardcoded_variable_size_input()
for i in range(len(X_list)):
print(f'data point i = {i}')
## get data
x = X_list[i]
print(x.size())
## create cnn
_,C,_,_ = x.size()
cnn = get_cnn(C)
# pass data
y = cnn(x)
## make sure input and output have same size
assert x.size() == y.size(), f'Error: {x.size()} != {y.size()}'
##
if __name__ == '__main__':
#do_conv()
print()
print('start main')
any_input_output_equals_input()
print('Done \a')
Maybe something like this could work?也许这样的事情可以工作?
import torch
import torch.nn
class PaddedConv2d(nn.Module):
"""
A simple class to perform convolutions with padding so that input and
output size is the same
"""
def __init__(self, conv2d, pad2d_type):
"""
Parameters
---
* conv2d : torch.nn.Conv2d
a convolutional layer used in this PaddedConv2d
* pad2d_type : type
a padding layer from torch.nn. I don't want the
instance itself since it needs the padding size to be created;
instead, I want the callable which returns an instance of
a padding layer, which will be created on the fly during the
"forward" pass.
"""
super().__init__(self)
self.conv2d = conv2d
self.pad2d_type = pad2d_type
def forward(self, in):
"""
Parameters
---
* in : torch.Tensor
the input tensor to be padded and then convolved. Shape (batch_size, channels, rows, cols)
"""
# computing padding size:
pad_h = torch.ceil((self.conv2d.kernel[0] - in.shape[2] * (1 - self.conv2d.stride[0]) - self.conv2d.stride[0]) / 2)
pad_w = torch.ceil((self.conv2d.kernel[1] - in.shape[3] * (1 - self.conv2d.stride[1]) - self.conv2d.stride[1]) / 2)
padder = self.pad2d_type((pad_w, pad_w, pad_h, pad_h))
return self.conv2d(padder(in))
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.