[英]How to access weight and L2 norm of conv layers in a CNN in Pytorch?
Are there PyTorch functions to access those?是否有 PyTorch 函数可以访问这些函数?
You can do it using您可以使用
torch.div(model[i].weight, torch.norm(model[i].weight), out=model[i].weight)
Toy example (documented inline).玩具示例(内联记录)。
import torch
from torch.nn import Linear, ReLU, CrossEntropyLoss, Sequential, Conv2d, MaxPool2d, Module, Softmax, BatchNorm2d, Dropout
from torch.optim import Adam
# Define model
model = Sequential(
Conv2d(1, 4, kernel_size=3, stride=1, padding=1),
BatchNorm2d(4),
ReLU(inplace=True),
MaxPool2d(kernel_size=2, stride=2),
# Defining another 2D convolution layer
Conv2d(4, 4, kernel_size=3, stride=1, padding=1),
BatchNorm2d(4),
ReLU(inplace=True),
MaxPool2d(kernel_size=2, stride=2),
)
optimizer = Adam(model.parameters(), lr=0.07)
criterion = CrossEntropyLoss()
# Train loop
for epoch in range(10):
optimizer.zero_grad()
# Forward
# y_hat = model(X_train)
# loss = criterion(y_train, y_hat)
# Backward
# loss.backward()
# optimizer.step()
# Now maunually update the weights
for i in range(len(model)):
with torch.no_grad():
if hasattr(model[i], 'weight'):
torch.div(model[i].weight, torch.norm(model[i].weight), out=model[i].weight)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.