Poor performance transfer learning ResNet50

Question

I have a dataset of 11k images labeled for semantic segmentation. About 8.8k belong to 'group 1' and the rest to 'group 2'

I am trying to simulate what would happen if we lost access to 'group 1' imagery but not a network trained from them.

So I trained ResNet50 on group 1 only. Then used that network as a starting point for training group 2 only.

Results are essentially slightly better than not training with group 2 imagery (3% in average per class accuracy) but less than 1% better than if I just started with imagenet weights. I tested freezing blocks of resnet50 and a range of learning rates.

Group 1 and 2 are part of the same problem domain but are a bit different. They are taken at different regions (in fact the whole set covers a bunch of areas but group 1 and 2 are disjoint in this regard) and a different camera/resolution. They are resized to a fixed size though this fixed size is closer to group 1 average size.

They are very different to imagenet images. They are monochrome, rectangular and are essentially one type of object that I'm segmenting.

I'm not seeking to get the same result as training on all the images at once but surely there must be a bump in doing this over just training from imagenet.

Answer 1

I have read a few articles about the same topic - i have 12k jpeg images from 3 classes and after 3 epochs the accuracy dropped to 0. I am awaiting delivery of a new graphics card to improve performance (it's currently taking 90 - 120 minutes per epoch) and hope to give more feedback. I am just wondering if the face that this model was designed for ImageNet and its 21k classes might be part of the problem - its too wide and deep, therefore too sensitive to changes to weights....... would be interested in others views