Intersection over Union over two segments

Question

Let's say I have this two Dataframes: Groundtruth and Prediction .

Each Dataframe has 3 columns; Action, Start and End .

**Prediction :**

Action  |   Start   | End
-------------------------
 3      |   0       | 10
 2      |   10      | 70
 3      |   80      | 120
 0      |  120      | 350
 7      |  400      | 610
...

**Groundtruth :**

Action  |   Start   | End
-------------------------
 2      |   20      | 140
 0      |  150      | 340
 6      |  420      | 600
...

I want to compute the Intersection-over-Union (IoU) over these two dataframes using all columns, meaning Action first to see if it's a correct prediction or not plus the start and the end segments for each action to see if starts and ends correctly.

Here's my code:

def compute_iou(y_pred, y_true):
    y_pred = y_pred.flatten()
    y_true = y_true.flatten()
    cm = confusion_matrix(y_true, y_pred)
    intersection = np.diag(cm)
    ground_truth_set = cm.sum(axis=1)
    predicted_set = cm.sum(axis=0)
    union = ground_truth_set + predicted_set - intersection
    IoU = intersection / union
    for i in range(len(IoU)):
        if (IoU[i]>0.5):
            IoU[i] = 1
    return round(np.mean(IoU)*100, 3)

This works when I want to calculate the IoU over the actions column.

Now how can I adapt this so I can get IoU to get the overlapping segments over the start and end columns?

PS: Groundtruth and Prediction dataframes don't have the same number of rows.

Answer 1

(post edit)

The calculation is broken into three cases:

• Overlap: where the activity matches, and there's an overlap between the ground truth interval and the prediction interval.
• No overlap: the activity matches, but there's no such overlap.
• No hit: the activity wasn't predicted at all, or there's a wrong activity.

Here's the code:

df = pd.merge(pred, groundtruth, on = "Action", how = "outer",  suffixes = ["_pred", "_gt"])

overlap = df[(df.Start_pred < df.End_gt) & (df.Start_gt < df.End_pred)]

intersection = (overlap[["End_pred", "End_gt"]].min(axis=1) - overlap[["Start_pred", "Start_gt"]].max(axis=1)).sum()

union_where_overlap = (overlap[["End_pred", "End_gt"]].max(axis=1) - overlap[["Start_pred", "Start_gt"]].\
                    min(axis=1)).sum()

no_hit = df[df.isna().sum(axis=1) > 0]
union_no_hit = (no_hit[["End_pred", "End_gt"]].max(axis=1) - no_hit[["Start_pred", "Start_gt"]].min(axis=1)).sum()

no_overlap = df[~((df.Start_pred < df.End_gt) & (df.Start_gt < df.End_pred))].dropna()
union_no_overlap = ((no_overlap.End_pred - no_overlap.Start_pred) + (no_overlap.End_gt - no_overlap.Start_gt)).sum()

IoU = intersection / (union_no_hit + union_where_overlap + union_no_overlap)