在 Python 中自定义梯度下降算法的每一步中存储参数值

Question

I'm trying to make custom gradient descent estimator, however, I am encountering the issue with storing the parameter values at every step of the gradient descent algorithm. Here is the code skeleton:

from numpy import *
import pandas as pd
from joblib import Parallel, delayed
from multiprocessing import cpu_count

ftemp = zeros((2, )) 
stemp = empty([1, ], dtype='<U10') 
la = 10

vals = pd.DataFrame(index=range(la), columns=['a', 'b', 'string']

def sfun(k1, k2, k3, string):
    a = k1*k2
    b = k2*k3
    s = string

    nums = [a, b]
    strs = [s]

    return(nums, strs)

def store(inp):
    r = rsfun(inp[0], inp[1], inp[2], inp[3])

    ftemp = append(ftemp, asarray(r[0]), axis = 0)
    stemp = append(stemp, asarray(r[1]), axis = 0)
    
    return(ftemp, stemp)

for l in range(la):
    inputs = [(2, 3, 4, 'he'),
              (4, 6, 2, 'je'), 
              (2, 7, 5, 'ke')]

    Parallel(n_jobs = cpu_count)(delayed(store)(i) for i in inputs)

    vals.iloc[l, 0:2] = ftemp[0, 0], ftemp[0, 1]
    vals.iloc[l, 2] = stemp[0]

    d = ftemp[2, 0]-ftemp[0, 0]
    

Note: most of the gradient descent stuff is removed because I do not have any issues with that. the main issues that I have are storing the values at each step.

sfun() is the loss function (I know that it doesn't look like that here) and store() is just an attempt to store the parameter values with each step.

The important aspect here is that I want to parallelize the process as sfun() is computationally expensive and the issue with that I want to save values for all parallel runs.

I tried solving this in many different ways, but I always get a different error.

Answer 1

No need to make a temporary storage array, possible to store the results of Parallel() function directly by:

a = Parallel(n_jobs = cpu_count)(delayed(store)(i) for i in inputs)

Most importantly, a is populated in order that the inputs are given.