使用** kwargs在函数之间传递值

Question

I currently have one function run_prob that takes my models and returns the y_prob. It works perfectly. However, below, I have put together another two functions pred_prob and cal_prob , and I would like these two functions to work together to print out my pred_prob,count,true_prob into a new dataframe.

I know that def pred_prob_ and def cal_prob_ do not work. However, the code between the functions is operational.

The issue that I'm having is that I'm not sure how to set run_prob_cv to pass my other models that I have built into them (See below).

def run_prob_cv(X, y, clf_class, **kwargs):
    kf = KFold(len(y), n_folds=5, shuffle=True)
    y_prob = np.zeros((len(y),2))
    for train_index, test_index in kf:
        X_train, X_test = X[train_index], X[test_index]
        y_train = y[train_index]
        clf = clf_class(**kwargs)
        clf.fit(X_train,y_train)
        # Predict probabilities, not classes
        y_prob[test_index] = clf.predict_proba(X_test)
    return y_prob

The models that I have built and their names are: KNN, SVM, RF, and GNB

I was thinking that the functions would flow something like this:

def pred_prob_(y_prob, run_prob_cv(X, y, MODEL, n_estimators, **kwargs):
# Use 10 estimators so predictions are all multiples of 0.1
    pred_prob = run_prob_cv(X, y, MODEL, n_estimators=10)
    pred_churn = pred_prob[:,1]
    is_churn = y == 1
    # Number of times a predicted probability is assigned to an observation
    counts = pd.value_counts(pred_churn)
    return counts
def cal_prob_(counts):
    # calculate true probabilities
    true_prob = {}
    for prob in counts.index:
        true_prob[prob] = np.mean(is_churn[pred_churn == prob])
        true_prob = pd.Series(true_prob)
    # pandas-fu
    counts = pd.concat([counts,true_prob], axis=1).reset_index()
    counts.columns = ['pred_prob', 'count', 'true_prob']
    #counts
    counts.sort_index(by=['pred_prob','true_prob'], ascending=[False, True])

The end goal is to be able to take my four models and run them like this without having to generate repetitive code:

pred_prob_(y_prob, run_prob_cv(X, y, KNN, n_estimators, **kwargs)
pred_prob_(y_prob, run_prob_cv(X, y, RF,  n_estimators, **kwargs)
pred_prob_(y_prob, run_prob_cv(X, y, SVM, n_estimators, **kwargs)
pred_prob_(y_prob, run_prob_cv(X, y, GNB,  n_estimators, **kwargs)

Answer 1

def pred_prob(X, y, MODEL):
    pred_prob = run_prob_cv(X, y, MODEL)
    pred_churn = pred_prob[:,1]
    is_churn = y == 1
    # Number of times a predicted probability is assigned to an observation
    counts = pd.value_counts(pred_churn)
    # calculate true probabilities
    true_prob = {}
    for prob in counts.index:
        true_prob[prob] = np.mean(is_churn[pred_churn == prob])
        true_prob = pd.Series(true_prob)
    # pandas-fu
    counts = pd.concat([counts,true_prob], axis=1).reset_index()
    counts.columns = ['pred_prob', 'count', 'true_prob']
    #counts
    print counts.sort_index(by=['pred_prob','true_prob'], ascending=[False, True])

pred_prob(X, y, KNN)
pred_prob(X, y, RF)
pred_prob(X, y, GNB)
pred_prob(X, y, SVC)

Answer 2

The **kwargs syntax is used to collect zero or more named parameters and manage them as a dict . Similarly, the *posargs syntax does the same things with unnamed positional parameters , making them available within the function as a tuple .

You can, for example, call a function like this:

foo(1, 2, 3, namedparam1=4, namedparam2=5)

If you like, you can explicitly receive the named parameters:

def foo(x, y, z, namedparam1, namedparam2):

Or you can "glob them together" using ** :

def foo(x, y, z, **kwargs):

In this case, you can inspect, edit, or modify the resulting dict:

    if 'zanzibar' not in kwargs:
        kwargs['zanzibar'] = (-6.167904, 39.228628)

You can also pass the dict to another function, as named arguments, using the exact same ** syntax:

    bar(z, y, x, **kwargs)

If bar has explicitly named parameters matching the keys in kwargs , they will receive the values in the dict. If bar does not have named parameters matching the keys in dict, then if bar has a ** catch-all dictionary, the values will go there, otherwise an exception will be thrown.

For example:

def bar(z, y, x, zanzibar, **kwargs):

In this example, zanzibar is now a required parameter. You can pass it via **kwargs or by spelling it out in your call, but it must be present somehow. Other named parameters in **kwargs are still optional.