Python Numpy浮点数组精度

Question

I am trying to solve for SVM optimisation problem using Pegasos mini-batch algorithm (as in Fig 2) from this link: http://www.cs.huji.ac.il/~shais/papers/ShalevSiSrCo10.pdf

#X: m*n matrix with m examples and n features per example (m=4000 and n=784 in my case), Y: m length vector containing 1 or -1 for each example, l: lambda as given in algorithm (l=1 in my code), itr: number of iterations, k: size of batch (100) in my case
def pegasos(X,Y,l,n,m,itr,k):
w = np.zeros((1,n),dtype=np.float32)
print m, n
diff = 0.0
for t in range(1,itr+1):
    A = random.sample(range(1,m),k)
    total = np.zeros((1,n),dtype=np.float32)
    eta = 1/(l*t)
    for i in A:
        x = X[i]
        y = Y[i]
        p = y*(np.dot(w,x.T))
        if p < 1:
            p1 = y*x
            total = np.add(total,p1)
    #update rule
    w = np.add((w*(1-(1/t))) , (eta*total*(1/k)))
return w

My dataset is in such a way that when my variable total is computed, I get mostly 0s but there are a few values in the order of 10^(-1) to 10^(-5). As soon as total is multiplied by (eta/k) at the update rule, all the values become 0. and hence at every iteration the w I obtain is 0. which should not be the case. I have tried ways to increase the precision of my floats but they don't seem to work at all. When I use basic Pegasos algorithm (as given in Fig 1 in the above link), I don't face any problem, thus my dataset is not utterly weird. Any help regarding this issue would be highly appreciated :)

Answer 1

If you need precision, you should use np.float64 (the normal floating point precision double ).

If you are using Python 2, you are using integer divition in (1/t) , (1/k) , and (1/l) . Write it as 1.0/ , to force a floating point division.