What kind of data stored in pre-trained model, such as caffe model zoo?

Question

I came across this question from reading squeeze net paper. The authors state that they use Deep Compression to compress the pre-trained model. The algorithm includes Huffman Code etc.

I infer the pre-trained are all parameters and I do know these parameters are generated when training the network but I have no idea how the parameters are generated. What role do parameters of the pre-trained model play when doing prediction?

It sounds to me like black magic

Answer 1

The pre-trained model consists of the weights for all of the layer connections to/from every kernel of every layer. That's the "heavy lifting" from the first 40-80 epochs of training. It should be ready to do predictions, or continue with whatever fine-tuning you'd care to apply.

It's not really black magic. Each framework has a facility to dump (back-up) the parameter values at specified intervals and at completion of training. Granted, these are relatively large files -- hence the use of compression. Each framework has a facility to read in such a dump file in order to bootstrap a model.