在评估等于2个函数调用的结果时，寻找一种方法（理想情况下在Python中）以获取第二个参数值

Question

我有两个函数， g1(x,y)和g2(x,y)返回一个浮点数

例如， g1(1,2) - >返回0.345
g2(1,2) - >返回0.453

现在，为了绘制决策边界，我想满足：
g2(x,y) == g1(x,y) ，
或者重新安排为：
g1(x,y) - g2(x,y) == 0

如果我生成一系列x值1,2,3,4,5 ，我怎样才能找到产生g1(x,y) - g2(x,y) == 0的相应y值？

我真的不知道该怎么做，并会欣赏任何想法。 你认为scipy.optimize.minimize会是一个好方法吗？ 如果是这样，我将如何做到这一点（我试过并尝试使用语法失败）。

谢谢你的帮助！

编辑：

你问了g1（）和g2（）的等式，这里是:)

$ \\ Rightarrow g_1（\\ pmb {x}）= \\ pmb {x} ^ {\\，t} - \\ frac {1} {2} \\ Sigma_1 ^ { - 1} \\ pmb {x} + \\ bigg（\\ Sigma_1 ^ { - 1} \\ pmb {\\ mu} {\\，1} \\ bigg）^ t + \\ bigg（ - \\ frac {1} {2} \\ pmb {\\ mu} {\\，1} ^ {\\，t } \\ Sigma_ {1} ^ { - 1} \\ pmb {\\ mu} _ {\\，1} - \\ frac {1} {2} ln（| \\ Sigma_1 |）\\ bigg）\\ \\ quad g_2（\\ pmb { x}）= \\ pmb {x} ^ {\\，t} - \\ frac {1} {2} \\ Sigma_2 ^ { - 1} \\ pmb {x} + \\ bigg（\\ Sigma_2 ^ { - 1} \\ pmb { \\ mu} {\\，2} \\ bigg）^ t + \\ bigg（ - \\ frac {1} {2} \\ pmb {\\ mu} {\\，2} ^ {\\，t} \\ Sigma_ {2} ^ { -1} \\ pmb {\\ mu} _ {\\，2} - \\ frac {1} {2} ln（| \\ Sigma_2 |）\\ bigg）$

（嗯，不知怎的，Latex不工作，我会将其上传为图片）：

这就是我实现它们的方式：

def discriminant_function(x_vec, cov_mat, mu_vec):
    """
    Calculates the value of the discriminant function for a dx1 dimensional
    sample given covariance matrix and mean vector.

    Keyword arguments:
        x_vec: A dx1 dimensional numpy array representing the sample.
        cov_mat: numpy array of the covariance matrix.
        mu_vec: dx1 dimensional numpy array of the sample mean.

    Returns a float value as result of the discriminant function.

    """
    W_i = (-1/2) * np.linalg.inv(cov_mat)
    assert(W_i.shape[0] > 1 and W_i.shape[1] > 1), 'W_i must be a matrix'

    w_i = np.linalg.inv(cov_mat).dot(mu_vec)
    assert(w_i.shape[0] > 1 and w_i.shape[1] == 1), 'w_i must be a column vector'

    omega_i_p1 = (((-1/2) * (mu_vec).T).dot(np.linalg.inv(cov_mat))).dot(mu_vec)
    omega_i_p2 = (-1/2) * np.log(np.linalg.det(cov_mat))
    omega_i = omega_i_p1 - omega_i_p2
    assert(omega_i.shape == (1, 1)), 'omega_i must be a scalar'

    g = ((x_vec.T).dot(W_i)).dot(x_vec) + (w_i.T).dot(x_vec) + omega_i
    return float(g)

为了分类我写的数据：

进口经营者

def classify_data(x_vec, g, mu_vecs, cov_mats):
    """
    Classifies an input sample into 1 out of x classes determined by
    maximizing the discriminant function g_i().

    Keyword arguments:
        x_vec: A dx1 dimensional numpy array representing the sample.
        g: The discriminant function.
        mu_vecs: A list of mean vectors as input for g.
        cov_mats: A list of covariance matrices as input for g.

    Returns a tuple (g_i()_value, class label).

    """
    assert(len(mu_vecs) == len(cov_mats)), 'Number of mu_vecs and cov_mats must be equal.'

    g_vals = []
    for m,c in zip(mu_vecs, cov_mats): 
        g_vals.append(g(x_vec, mu_vec=m, cov_mat=c))

    max_index, max_value = max(enumerate(g_vals), key=operator.itemgetter(1))
    return (max_value, max_index + 1)

并且代码到目前为止用于分类，例如，

import prettytable

classification_dict, error = empirical_error(all_samples, [1,2], classify_data, [discriminant_function,\
        [mu_est_1, mu_est_2],
        [cov_est_1, cov_est_2]])

labels_predicted = ['w{} (predicted)'.format(i) for i in [1,2]]
labels_predicted.insert(0,'training dataset')

train_conf_mat = prettytable.PrettyTable(labels_predicted)
for i in [1,2]:
    a, b = [classification_dict[i][j] for j in [1,2]]
    # workaround to unpack (since Python does not support just '*a')
    train_conf_mat.add_row(['w{} (actual)'.format(i), a, b])
print(train_conf_mat)
print('Empirical Error: {:.2f} ({:.2f}%)'.format(error, error * 100))


+------------------+----------------+----------------+
| training dataset | w1 (predicted) | w2 (predicted) |
+------------------+----------------+----------------+
|   w1 (actual)    |       49       |       1        |
|   w2 (actual)    |       1        |       49       |
+------------------+----------------+----------------+
Empirical Error: 0.02 (2.00%)

对于像这样的简单数据集：

编辑：

对于协方差相等的简单情况（线性决策边界），我能够使用fsolve函数：

from scipy.optimize import fsolve
x = list(np.arange(-2, 6, 0.1))
y = [fsolve(lambda y: discr_func(i, y, cov_mat=cov_est_1, mu_vec=mu_est_1) - \
                 discr_func(i, y, cov_mat=cov_est_2, mu_vec=mu_est_2), 0) for i in x]

但是，对于二次解决方案，它不起作用

/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/scipy/optimize/minpack.py:236: RuntimeWarning: The iteration is not making good progress, as measured by the 
  improvement from the last five Jacobian evaluations.
  warnings.warn(msg, RuntimeWarning)

任何提示或替代方案？

EDIT2：

我能够通过scipy.optimize.bisect （模拟到fsolve）解决它。 结果看起来是“正确的” - 我解决了一个更简单的情况，其中决策边界是线性函数（x2 = 3-x1），当我使用bisect ，它计算了精确的结果，例如，x1 = 3和x2 = 3。

无论如何，这里是二次函数的结果（我通过这里通过最大似然估计估计参数）和具有相等协方差的线性情形！ 非常感谢您的时间和帮助！

对于

from matplotlib import pyplot as plt
import numpy as np
import scipy.optimize

x = np.arange(-6,6,0.1)
true_y = [true_dec_bound(x1) for x1 in x]



for i in [50,1000,10000]:

    # compute boundary for MLE estimate
    y_est = []
    for j in x:
        y_est.append(scipy.optimize.bisect(lambda y: discr_func(j, y, cov_mat=cov1_ests[i], mu_vec=mu1_ests[i]) - \
                      discr_func(j, y, cov_mat=cov2_ests[i], mu_vec=mu2_ests[i]), -10, 10))
    y_est = [float(i) for i in y_est]

    # plot data
    f, ax = plt.subplots(figsize=(7, 7))
    plt.ylabel('$x_2$', size=20)
    plt.xlabel('$x_1$', size=20)
    ax.scatter(samples_c1[i][:,0], samples_c1[i][:,1], \
           marker='o', color='green', s=40, alpha=0.5, label='$\omega_1$')
    ax.scatter(samples_c2[i][:,0], samples_c2[i][:,1], \
           marker='^', color='red', s=40, alpha=0.5, label='$\omega_2$')
    plt.title('%s bivariate random training samples per class' %i)
    plt.legend()

    # plot boundaries
    plt.plot(x_true50, y_true50, 'b--', lw=3, label='true param. boundary')
    plt.plot(x_est50, y_est50, 'k--', lw=3, label='MLE boundary')

    plt.legend(loc='lower left')
    plt.show()

Answer 1

我想暂时发布我的暂定解决方案。 但它可能不是最佳的......

def discr_func(x, y, cov_mat, mu_vec):
    """
    Calculates the value of the discriminant function for a dx1 dimensional
    sample given covariance matrix and mean vector.

    Keyword arguments:
        x_vec: A dx1 dimensional numpy array representing the sample.
        cov_mat: numpy array of the covariance matrix.
        mu_vec: dx1 dimensional numpy array of the sample mean.

    Returns a float value as result of the discriminant function.

    """
    x_vec = np.array([[x],[y]])

    W_i = (-1/2) * np.linalg.inv(cov_mat)
    assert(W_i.shape[0] > 1 and W_i.shape[1] > 1), 'W_i must be a matrix'

    w_i = np.linalg.inv(cov_mat).dot(mu_vec)
    assert(w_i.shape[0] > 1 and w_i.shape[1] == 1), 'w_i must be a column vector'

    omega_i_p1 = (((-1/2) * (mu_vec).T).dot(np.linalg.inv(cov_mat))).dot(mu_vec)
    omega_i_p2 = (-1/2) * np.log(np.linalg.det(cov_mat))
    omega_i = omega_i_p1 - omega_i_p2
    assert(omega_i.shape == (1, 1)), 'omega_i must be a scalar'

    g = ((x_vec.T).dot(W_i)).dot(x_vec) + (w_i.T).dot(x_vec) + omega_i
    return float(g)

#g1 = discr_func(x, y, cov_mat=cov_mat1, mu_vec=mu_vec_1)
#g2 = discr_func(x, y, cov_mat=cov_mat2, mu_vec=mu_vec_2)

x_est50 = list(np.arange(-6, 6, 0.1))
y_est50 = []
for i in x_est50:
    y_est50.append(scipy.optimize.bisect(lambda y: discr_func(i, y, cov_mat=cov_est_1, mu_vec=mu_est_1) - \
                      discr_func(i, y, cov_mat=cov_est_2, mu_vec=mu_est_2), -10,10))
y_est50 = [float(i) for i in y_est50]

结果如下:(蓝色的二次情形，红色的线性情况（等方差）