向量化多元正态 pdf python (PyTorch/NumPy)

Question

I have N Gaussian distributions (multivariate) with N different means (covariance is the same for all of them) in D dimensions.

I also have N evaluation points, where I want to evaluate each of these (log) PDFs.

This means I need to get a NxN matrix, call it "kernels". That is, the (i,j)-th entry is the j-th Gaussian evaluated at the i-th point. A naive approach is:

from torch.distributions.multivariate_normal import MultivariateNormal
import numpy as np

# means contains all N means as rows and is thus N x D
# same for eval_points
# cov is not a problem , just a DxD matrix that is equal for all N Gaussians 

kernels = np.empty((N,N))
    for i in range(N):
        for j in range(N):
            kernels[i][j] = MultivariateNormal(means[j], cov).log_prob(eval_points[i])

Now one for loop we can get rid of easily, since for example if we wanted all the evaluations of the first Gaussian, we simply do:

MultivariateNormal(means[0], cov).log_prob(eval_points).squeeze()

and this gives us a N x 1 list of values, that is the first Gaussian evaluated at all N points.

My problem is that, in order to get the full N x N matrix, this doesn't work:

kernels = MultivariateNormal(means, cov).log_prob(eval_points).squeeze()

It doesn't figure out that it should evaluate each mean with all evaluation points in eval_points, and it doesn't return a NxN matrix with these which would be what I want. Therefore, I am not able to get rid of the second for loop, over all N Gaussians.

Answer 1

You are passing wrong shaped tensors to MultivariateNormal 's constructor. You should pass a collection of mean vectors of shape (N, D) and a collection of precision matrix cov of shape (N, D, D) for N D -dimensional gaussian.

You are passing mu of shape (N, D) but your precision matrix is not well-shaped. You will need to repeat the precision matrix N number of times before passing it to the MultivariateNormal constructor. Here's one way to do it.

N = 10
D = 3
# means contains all N means as rows and is thus N x D
# same for eval_points
# cov is not a problem , just a DxD matrix that is equal for all N Gaussians 

mu = torch.from_numpy(np.random.randn(N, D))
cov = torch.from_numpy(make_spd_matrix(D, D))
cov_n = cov[None, ...].repeat_interleave(N, 0)

assert cov_n.shape == (N, D, D)

kernels = MultivariateNormal(mu, cov_n)