如何将batch_normalization从python转换为c#

Question

In the last few months, I worked through Python and Tensorflow, building a neural network. The network performs pretty well on a large amount of data (the precision of my prediction is 85% after training on a data-set of 120000 records).

My neural network makes use of batch-normalization, learning-rate decay, dropout.. It uses an Adam-Optimizer to minimize the loss. After training I store my model by a saver to store the mean/variance-variables of batch_normalization:

    saver = tf.compat.v1.train.Saver(tf.global_variables())
    saver.save(sess, "sessionSave")

After searching for a proper way to convert this model to c#, I found a Tensorflow- implementation for .NET (SciSharp). But I cannot find an implementation of batch_normalization. In this specific case it would be the following python code call which I need to convert:

    Z_BN = tf.contrib.layers.batch_norm(Z, is_training=train,updates_collections=ops.GraphKeys.UPDATE_OPS, scope="scope"+str(i), reuse=True)

If there is a way to convert this call, I would need to solve the problem of the saved mean/variance-variables differently as well. I think, I am not able to implement batch_normalization myself. Is there someone who can provide an implementation for this requirement?

Answer 1

If you only need your model to work in .NET for inference, it should be sufficient to convert it to some widely used format such as ONNX, which you can later use with ML.NET: https://docs.microsoft.com/en-us/dotnet/machine-learning/tutorials/object-detection-onnx

You may also try to use the model directly via ML.NET, but your mileage may vary: https://docs.microsoft.com/en-us/dotnet/machine-learning/tutorials/image-classification

If neither of these approaches work for you, or you need to fine-tune/train the model from C# too, you can try a commercial solution: Gradient , which exposes the entire TensorFlow 1.15 APIs to .NET. Though in that case you would have to use the supported BatchNormalization class from either tensorflow.keras.layers or tensorflow.layers (the later one is deprecated) instead of the one from contrib . The later is a wrapper anyway .

Disclaimer: I am the author of Gradient.