How can I make dot product of vector and matrix faster in MATLAB?

Question

My code works however it's fairly slow and i need to run it multiple times so it's very inefficient. i am positive there is a more efficient way of calculating it.

The code is an implementation of this equation:

where k(x,y) is the dot product of the two vectors xi and yj are the rows i,j of the two matrices A and B, respectively.

I'd like to also note that the number of rows in each matrix is in the thousands.

here is my code

m=size(A,1);
Kxx=0;
for i=1:m
    x=A(i,:);
    X=A(i+1:end,:);
    Kxx=Kxx+2*sum(dot(ones(m-i,1)*x,X,2));
end
Kxx=Kxx/(m*(m-1));

n=size(B,1);
Kyy=0;
for j=1:n
   y=B(j,:);
   YY=B(j+1:end,:);
   Kyy=Kyy+2*sum(dot(ones(n-j,1)*y,YY,2));
end
Kyy=Kyy/(n*(n-1));

Kxy=0;
for i=1:m
    x=A(i,:);
   for j=1:n
       y=B(j,:);
       Kxy=Kxy+dot(x,y);
   end
end
Kxy=Kxy*2/(m*n);

Dxy=Kxx+Kyy-Kxy;

Answer 1

Your edit makes our jub much easier. Here's what you just have to do for a fully vectorized solution:

C=A*A';  %'
Kxx=sum(sum(C-diag(diag(C))))/m/(m-1);
C=B*B';  %'
Kyy=sum(sum(C-diag(diag(C))))/n/(n-1);
Kxy=2*mean(reshape(A*B.',[],1));   %'

Dxy=Kxx+Kyy-Kxy;

Thanks to @hiandbaii for pointing out that the equivalent of dot for complex vectors involves the conjugate transpose rather than the transpose.

---

Original, looping version for historical sentimental reasons:

I'm not sure whether the first two loops can be vectorized without huge memory overhead. So while I figure this out, here's a version in which the first two loops are a bit simplified, and the third loop is replaced by a vectorized operation:

%dummy input data
A=rand(5);
B=rand(5);

m=size(A,1);
Kxx=0;
for l=1:m
    x=A(l,:);
    X=A(l+1:end,:);
    Kxx=Kxx+2*sum(X*x.');         %'
end
Kxx=Kxx/(m*(m-1));

n=size(B,1);
Kyy=0;
for l=1:n
   y=B(l,:);
   YY=B(l+1:end,:);
   Kyy=Kyy+2*sum(YY*y.');         %'
end
Kyy=Kyy/(n*(n-1));

Kxy=2*mean(reshape(A*B.',[],1));  %'

Dxy=Kxx+Kyy-Kxy;