pytorch modifying pretrained model, will the old weight get re-initialized?

Question

I'm modifying a pretrained efficient-net model in pytorch. I'm doing the following in order:

1. Create the default model, load the imagenet weights.

2. Then, change the number of channels in the first layer, and delete few layers while adding few.

from efficientnet_pytorch import EfficientNet
from efficientnet_pytorch.utils import Conv2dStaticSamePadding

PATH = "../input/efficientnet-pytorch/efficientnet-b0-08094119.pth"
model = EfficientNet.from_name('efficientnet-b0')
model.load_state_dict(torch.load(PATH))

# augment model with 4 channels

model._conv_stem = Conv2dStaticSamePadding(4, 32, kernel_size = (3,3), stride = (2,2), 
                                                             bias = False, image_size = 512)

model._fc = torch.nn.Linear(in_features=1280, out_features=2, bias=True)

My question is: what will happen to the original weights that I loaded? Will it be there or all of the model will get randomly initialized?

Answer 1

If you are re-defining some of the layers, which you seem to be doing with model._conv_stem and model._fc . Then, yes those will be randomly initialized, thus meaning the loaded weights will no longer be used on those layers. The rest of the model will of course stay untouched and will use the loaded weights.