Access lower layer output in higher layer using CNTK and transfer learning

Question

I am searching for a way to forward lower layer output to a higher layer with a loaded VGG16 model using CNTK.

The background of my problem is:

I reimplemented some parts of Fully Convolutional Networks for Semantic Segmentation but then I ran into some problems: Starting with this example I first replaced the fully connected layers with fully convolutional and slit the sequence in the model definition part into chunks where I could simply access pool3 and pool4 for the later usage in eg. Convolution2D((1,1), num_classes, name='score_pool4')(pool4) . This works fine but after building the model I noticed, that I need to implement an own way to read batches because the build-in reader does not support 2D labels right now. Now I simply read the images using OpenCV and replaced the training_session(...).train() with a for loop and trainer.train_minibatch({model['features']: my_loaded_features, model['labels']: my_2D_labels}) this works well but because of the removed training_session part I don't know where I could apply the existing VGG16 weights.

My problem is:

I searched for transfer learning examples where those guys load models using C.load_model(...) and then clone the needed layers but now I am wondering how could I access cloned_layers->pool4 (in the middle of the loaded model) if I also want to use it in deeper layers.

I tried Convolution2D((1,1), num_classes, name='score_pool4')(cloned_layers.find_by_name('pool4')) but I ended up with some error messages while learner initialization because of "unknown shape information" in used weight variables.

So how can I access those layers within the loaded model for later (deeper) usage?

Thanks for reading (and maybe helping)!

Answer 1

If you are looking to read custom data. There are two tutorials on building your own readers. https://cntk.ai/pythondocs/manuals.html

Regarding cloning parts of a network - here is a link to another post on StackOverflow that has exemplar code