How to calculate mean average precision (mAP) using TensorFlow?

Question

I want to use TensorFlow to calculate hashcode's mAP （mean average precision）， but I don't know how to use tensor calculations directly.

The code which using NumPy is the following:

import numpy as np
import time
import os
# read train and test binarayCode
CURRENT_DIR = os.getcwd()



def getCode(train_codes,train_groudTruth,test_codes,test_groudTruth):

    line_number = 0
    with open(CURRENT_DIR+'/result.txt','r') as f:
        for line in f:
            temp = line.strip().split('\t')
            if line_number < 10000:
                test_codes.append([i if i==1 else -1  for i in map(int, list(temp[0]))])
                list2 = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
                list2[int(temp[1])] = 1
                test_groudTruth.append(list2) # get test ground truth(0-9)
            else:
                train_codes.append([i if i==1 else -1  for i in map(int, list(temp[0]))]) # change to -1, 1
                list2 = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
                list2[int(temp[1])] = 1
                train_groudTruth.append(list2) # get test ground truth(0-9)

            line_number += 1
    print 'read data finish'

def getHammingDist(code_a,code_b):
    dist = 0
    for i in range(len(code_a)):
         if code_a[i]!=code_b[i]:
             dist += 1
    return dist 


if __name__ =='__main__':
    print getNowTime(),'start!'

    train_codes = []
    train_groudTruth =[]

    test_codes = []
    test_groudTruth = []
    # get g.t. and binary code
    getCode(train_codes,train_groudTruth,test_codes,test_groudTruth)
    train_codes = np.array(train_codes)
    train_groudTruth = np.array(train_groudTruth)
    test_codes = np.array(test_codes)
    test_groudTruth = np.array(test_groudTruth)
    numOfTest = 10000

    # generate hanmming martix, g.t. martix  10000*50000
    gt_martix = np.dot(test_groudTruth, np.transpose(train_groudTruth))
    print getNowTime(),'gt_martix finish!'
    ham_martix = np.dot(test_codes, np.transpose(train_codes)) # hanmming distance map to dot value 
    print 'ham_martix finish!'

    # sort hanmming martix,Returns the indices that would sort an array.
    sorted_ham_martix_index = np.argsort(ham_martix,axis=1)

    # calculate mAP
    print 'sort ham_matrix finished,start calculate mAP'

    apall = np.zeros((numOfTest,1),np.float64)
    for i in range(numOfTest):
        x = 0.0
        p = 0
        test_oneLine = sorted_ham_martix_index[i,:]
        length = test_oneLine.shape[0]
        num_return_NN = 5000 # top 1000
        for j in range(num_return_NN):
             if gt_martix[i][test_oneLine[length-j-1]] == 1: # reverse
                 x += 1
                 p += x/(j+1)
        if p == 0:
            apall[i]=0
        else:
            apall[i]=p/x

    mAP = np.mean(apall)
    print 'mAP:',mAP

---

I want to re-write the code above using tensor operations (like tf.equal() 、 tf.reduce_sum() so on).

for example

I want to calculate valid accuracy of images

logits = self._model(x_valid)
valid_preds = tf.argmax(logits, axis=1)
valid_preds = tf.to_int32(valid_preds)
self.valid_acc = tf.equal(valid_preds, y_valid)
self.valid_acc = tf.to_int32(self.valid_acc)
self.valid_acc = tf.to_float(tf.reduce_sum(self.valid_acc))/tf.to_float(self.batch_size)

---

I want to use TensorFlow to calculate hashcode's mAP （mean average precision） this way（like tf.XX opreation）

How could I do？ Thanks！

Answer 1

If you just want to calculate average precision based on the validation set predictions, you can use the vector of predicted probabilities and the vector of true labels in this scikit-learn function .

If you really want to use a tensorflow function, there's a tensorflow function average_precision_at_k .

For more info about average precision you can see this article .

Answer 2

You can just calculate the y_score (or predictions) and then use sklearn.metrics to calculate the average precision:

from sklearn.metrics import average_precision_score

predictions = model.predict(x_test)
average_precision_score(y_test, predictions)