比较两个标签向量相同的百分比

Question

Background

I have two clustering methods that I want to compare. I cluster my data objects with the one method, then with the other and label the objects for both methods. Now I would like to compare to what percentage the second method labels the data objects the same way as the first method.

Problem

I have data objects with two types of labels. The labels are integers without any intrinsic meaning other then those data objects with the same label (per label type) belong to the same group. I want to know to what percentage the two labellings are the same.

For example (pseudo-code where the == is element-wise):

>>> label1 = [1,1,1,1,2,2,2,3,3,3,3,3,4,4]
>>> label2 = [1,1,2,2,2,2,2,2,2,3,3,4,4,4]
>>> correctness = sum_of_true(label1 == label2) / 14
correctness: 9 / 14 = 0.6428571

However the labels might not used the same way. For example

>>> label1 = [1,1,1,1,2,2,2,3,3,3,3,3,4,4]
>>> label2 = [2,2,2,2,1,1,1,4,4,4,4,4,3,3]

are actually the same labelled and the correctness should be 1.0 .

For that I need to rename the label2 in such a way that the labels are as similar to label1 as possible.

Inefficient solution

An inefficient solution is to simply try to rename label2 in all possible solutions, calculate for each renaming the correctness as above in the example and take the solution with the best correctness. However the number of possible renames is the permutation of the number of labels. This can be a really huge number and makes this approach unusable.

Other solutions

I know about normalized mutual information (nmi) as a means to compare labels, but this is not what I am looking for. Reasons are that firstly nmi is not linear, secondly it is difficult to understand and communicate and thirdly I simply want something else ;-) - in this case to know about he number (~ percentage) of same labelled data objects. The reason I want this something else has something to do with the later application of the labels.

So for example

>>> label1 = [1,1,1,1]
>>> label2 = [1,2,3,4]

I still want this to be of correctness 1/4 . I do not want to discuss here whether that is smart or not. In my later application this is what I need.

Allowing merging

Additionally there is the issue that the number of labels may be different for different for label1 and label2 . For my application I might be actually useful to be lenient towards this, allowing merging of labels to one on either side. For example

>>> label1 = [1,1,1,1]
>>> label2 = [1,2,3,4]

would become correctness of 1 if it's lenient towards merging of label2 , while it would be 0.5 for

>>> label1 = [1,1,2,2]
>>> label2 = [1,2,3,4]

Question

How can I calculate the correctness efficiently for

1. No merging allowed.
2. Merging in the first label allowed.
3. Merging in the second label allowed.

where, surely, the solution for 2. and 3. would be the same.

Notes

• I am using python for implementation.
• Please tell me what tags to use for this question if you know. I am not sure.

Answer 1

There are several well-established methods to evaluate the similarity of two clustering results. They already solved the alignment problem, which gets worse if the number of clusters vary.

You should probably just use one of them, in particular:

1. Rand index
2. Adjusted Rand Index
3. Jaccard
4. Fowlkes-Mallows index