在Keras中对可训练参数进行排序

Question

When I have some trainable parameters, say layer.trainable_weights . I want to sort those weights before feed into other operations, is it possible for me to do that? Can I use something like

import tensorflow as tf
p = layer.trainable_weights
p = tf.sort(p)

or are there any particular ways in Keras?

I'm new to Keras and TensorFlow. Really appreciate if someone can answer my questions, thanks in advance!

---

EDIT: For "other operations", I want to feed those sorted trainable weights into another neural network, but that neural network is fixed (not trainable). So what I want to do is something like

import tensorflow as tf
p = model.layer[0].trainable_weights
p = tf.sort(p)

another_model.trainable = False
x = another_model(p)
# x is involved in the loss function of the original model

Hope this is clear, and hope anyone can help me! (also, can I just use x=another_model.predict(p) instead of x=another_model(p) above?)

Answer 1

Of course, you can do it. Since you have not stated clearly what these downstreaming operations are, it is more difficult to answer your question.

If you only want to do something to monitor the training process, eg monitoring a custom metric to measure the cumulative distribution function of the weight matrix of interest, feel free to use tf.sort (you may use K.stop_gradient(W) before you sort it to further ensure no gradient is flowed back from any downstreaming process).

If you want to do things other than monitoring, eg computing a custom regularization term to play a part in training optimization, you should rethink how to implement this part both efficiently and effectively, but directly using tf.sort does not help! Why? Because this function is not implemented with back-propagation, (you might want to double check on this to see whether the latest version supports this feature)