在使用Keras的模型时需要帮助

Question

My goal is to make an easy neural network fit by providing 2 verticies of a certain Graph and 1 if there's a link or 0 if there's none.

I fit my model, it gets loss of about 0.40, accuracy of about 83% during fitting. I then evaluate the model by providing a batch of all positive samples and several batches of negative ones (utilising random.sample). My model gets loss of ~0.35 and 1.0 accuracy for positive samples and ~0.46 loss 0.68 accuracy for negative ones.

My understanding of neural networks if extremely limited, but to my understanding the above means it theoretically always is right when it outputs 0 when there's no link, but can sometimes output 1 even if there is none.

Now for my actual problem: I try to "reconstruct" the original graph with my neural network via model.predict. The problem is I don't understand what the predict output means. At first I assumed values above 0.5 mean 1, else 0. But if that's the case the model doesn't even come close to rebuilding the original.

I get that it won't be perfect, but it simply returns value above 0.5 for random link candidates.

Can someone explain to me how exactly model.predict works and how to properly use it to rebuild my graph?

Answer 1

The model that you trained is not directly optimized wrt the graph reconstruction. Without loss of generality, for a N -node graph, you need to predict N choose 2 links. And it may be reasonable to assume that the true values of the most of these links are 0.

When looking into your model accuracy on the 0-class and 1-class, it is clear that your model is prone to predict 1-class, assuming your training data is balanced. Therefore, your reconstructed graph contains many false alarm links. This is the exact reason why the performance of your reconstruction graph is poor.

If it is possible to retrain the model, I suggest you do it and use more negative samples.

If not, you need to consider applying some post-processing. For example, instead of finding a threshold to decide which two nodes have a link, use the raw predicted link probabilities to form a node-to-node linkage matrix, and apply something like the minimum spanning tree to further decide what are appropriate links.