机器学习逻辑回归 L2 正则化

Question

I am trying to implement L2 regularization to my data I wrote my function like this

 def logicalregP3(xtr,ytr,learning_rate,iteration,lamda):
   
    m=xtrain.T.shape[1]
    n=xtrain.T.shape[0]
   
    W= np.zeros((n,1))
  
    B = 0
    cost_list = []
    for i in range (iteration):
        z= np.array(np.dot(W.T, xtr.T),dtype=np.float32)+B
     
        a= 1/(1 + np.exp(-z))

    
        cost=-(1/m)*np.sum(ytr.T*np.log(a)+(1-ytr.T)*np.log(1-a))+(lamda*np.sum(W))
        # Gradient Descent
        
        regular=(lamda/(2*m))*W
        dW=(1/m)*np.dot(a-ytr.T,xtr)+regular
        dB=(1/m)*np.sum(a-ytr.T)      
        W=W-learning_rate*dW.T
        B=B-learning_rate*dB
        print("cost ", i ," ", cost)
   
        cost_list.append(cost)
       
            
    return W,B,cost_list

Then I call the function with these values

iterations=2000
learning_rate=0.0015
lamda=0.00001
Wp3,Bp3,cost_listp3=logicalregP3(xtrain,ytrain,learning_rate, iterations,lamda)

the output of my code:

cost  0   0.6931471824645996
cost  1   2.6271848720002735
cost  2   2.5287339955351484
cost  3   2.461344902235327
cost  4   2.4144645825946607
cost  5   2.3812474056254485
cost  6   2.357244340813794
cost  7   2.339536539714837
cost  8   2.3261846933514745
cost  9   2.3158827015378978
cost  10   2.307739196600699
cost  11   2.3011379381265
cost  12   2.2956485144978704
cost  13   2.290966932838124
cost  14   2.28687647957537

when I draw the graph: this is the output for the cost:

Is the output of my code normal or I have problem at the gradient descent

Answer 1

Actually L2 regularisation is not lambda * np.sum(W) but it is lambda * np.linalg.norm(W, ord=2) . In your case I assume that the gradient descent works well but you can't check it because your costs don't represent the quantity which is minimized here.