Numpy找到兩個二維ndarray的協方差

Question

我是numpy的新手，我堅持這個問題。 我有兩個二維numpy數組，如

x = numpy.random.random((10, 5))
y = numpy.random.random((10, 5))

我想使用numpy cov函數來查找這兩個ndarray行的協方差。 即，對於上面的例子，輸出數組應該由10個元素組成，每個元素表示相應行的ndarray的協方差。 我知道我可以通過遍歷行並找到兩個1D陣列的協方差來做到這一點，但它不是pythonic。

Edit1：兩個數組的協方差表示0, 1索引處的元素。

Edit2：目前這是我的實現

s = numpy.empty((x.shape[0], 1))
for i in range(x.shape[0]):
    s[i] = numpy.cov(x[i], y[i])[0][1]

Answer 1

使用協方差的定義： E(XY) - E(X)E(Y) 。

import numpy as np
x = np.random.random((10, 5))
y = np.random.random((10, 5))

n = x.shape[1]
cov_bias = np.mean(x * y, axis=1) - np.mean(x, axis=1) * np.mean(y, axis=1))
cov_bias * n / (n-1)

請注意， cov_bias對應於numpy.cov(bias=True) 。

Answer 2

這是有效的，但是我不確定對於更大的矩陣x和y是否更快，調用numpy.cov(x, y)計算我們用numpy.diag丟棄的許多條目：

x = numpy.random.random((10, 5))
y = numpy.random.random((10, 5))

# with loop
for (xi, yi) in zip(x, y):
    print(numpy.cov(xi, yi)[0][1])

# vectorized
cov_mat = numpy.cov(x, y)
covariances = numpy.diag(cov_mat, x.shape[0])
print(covariances)

---

我也為nxn大小的矩陣做了一些時間：

import time
import numpy

def run(n):
    x = numpy.random.random((n, n))
    y = numpy.random.random((n, n))

    started = time.time()
    for (xi, yi) in zip(x, y):
        numpy.cov(xi, yi)[0][1]

    needed_loop = time.time() - started

    started = time.time()
    cov_mat = numpy.cov(x, y)
    covariances = numpy.diag(cov_mat, x.shape[0])
    needed_vectorized = time.time() - started
    print(
        f"n={n:4d} needed_loop={needed_loop:.3f} s "
        f"needed_vectorized={needed_vectorized:.3f} s"
    )

for n in (100, 200, 500, 600, 700, 1000, 2000, 3000):
    run(n)

我的慢速MacBook Air上的輸出是

n= 100 needed_loop=0.006 s needed_vectorized=0.001 s
n= 200 needed_loop=0.011 s needed_vectorized=0.003 s
n= 500 needed_loop=0.033 s needed_vectorized=0.023 s
n= 600 needed_loop=0.041 s needed_vectorized=0.039 s
n= 700 needed_loop=0.043 s needed_vectorized=0.049 s
n=1000 needed_loop=0.061 s needed_vectorized=0.130 s
n=2000 needed_loop=0.137 s needed_vectorized=0.742 s
n=3000 needed_loop=0.224 s needed_vectorized=2.264 s

所以收支平衡點大約是n=600

Answer 3

這里有一個使用covariance的定義，並受到corr2_coeff_rowwise啟發 -

def covariance_rowwise(A,B):
    # Rowwise mean of input arrays & subtract from input arrays themeselves
    A_mA = A - A.mean(-1, keepdims=True)
    B_mB = B - B.mean(-1, keepdims=True)

    # Finally get covariance
    N = A.shape[1]
    return np.einsum('ij,ij->i',A_mA,B_mB)/(N-1)

樣品運行 -

In [66]: np.random.seed(0)
    ...: x = np.random.random((10, 5))
    ...: y = np.random.random((10, 5))

In [67]: s = np.empty((x.shape[0]))
    ...: for i in range(x.shape[0]):
    ...:     s[i] = np.cov(x[i], y[i])[0][1]

In [68]: np.allclose(covariance_rowwise(x,y),s)
Out[68]: True

Answer 4

選擇cov（x，y）的對角矢量並展開dims：

numpy.expand_dims(numpy.diag(numpy.cov(x,y),x.shape[0]),1)