Pytorch model outputting labels it was not trained on

Question

I have a Faster R CNN detector which I've trained with pytorch lightning on a quite noisy, but large, dataset. I would expect that after 1 epoch of training, the model would only output labels in the dataset, in my case 0 to 56. However, it is giving me labels such as 64 and 89. What is going on here? Where is it coming up with these labels it was never trained on?

Can't share any code because this problem probably relates to my dataset, not my code. With the COCO pretrained model, it works fine.

Answer 1

The problem was not my data or my model. The problem is the pytorch nn.module load_state_dict() method. This method has a argument strict which is supposed to allow users to load a state_dict without the exact same weight keys, but it actually causes the loaded model to be completely wrong. I highly recommend against using strict=False when loading a model with load_state_dict() in pytorch.