Computation of Pseidoinverse with SVD in C++ using BLAS and LAPACKE

Question

I am trying to implement the pseudoinverse computation A* of a matrix in order to solve the Ax=b for a square nxn matrix A with dimensions in C++. The arithmetic formula for A* is through the SVD decomposition.

So first I compute SVD(A)=USV^T and then A*=VS U^T, where S is the inverse diagonal S where its non-zero element si becomes 1/si in S*. Finally i compute the solution x=A*b

However I am not getting the correct result. I am using LAPACKE interface for c++ and cblas for matrix multiplication. Here is my code:

double a[n * n] = {2, -1, 2,1};
double b[n]={3,4};
double u[n * n], s[n],vt[n * n];

int lda = n, ldu = n, ldvt = n;

int info = LAPACKE_dgesdd(LAPACK_COL_MAJOR, 'A', n, n, a, lda, s,
               u, ldu, vt, ldvt);




for (int i = 0; i < n; i++) {
        s[i] = 1.0 / s[i];       
}

const int a = 1;
const int c = 0;

double r1[n];
double r2[n];
double res[n];

//compute the  first multiplication s*u^T
cblas_dgemm( CblasColMajor,CblasNoTrans, CblasTrans, n, n, n, a, u, ldvt, s, ldu, c, r1, n);

//compute the second multiplication v^T^T=vs*u^T
cblas_dgemm( CblasColMajor,CblasTrans, CblasNoTrans, n, n, n, a, vt, ldvt, r1, ldu, c, r2, n);

//now that we have the pseudoinverse A* solve by multiplying with b.
cblas_dgemm( CblasColMajor,CblasNoTrans, CblasNoTrans, n, 1, n, a, r2, ldvt, b, ldu, c, res, n);

after the second cblas_dgemm it is expected to have A* the pseudoinverse in r2. However after comparing with matlab pinv I am not getting the same result. If I print r2 the result gives:

 0.25   0.50
 0.25   0.50

but it should be

0.25   -0.50
0.25   0.50

Answer 1

The argument S of LAPACKE_dgesdd() represents the singular values of the matrix in the SVD decomposition . While it is of length n , it does not depict a vector as it represents a diagonal matrix. Indeed, the outcome of Su^T is a matrix of size n*n .

The routine cblas_dscal() can be applied in a loop to compute the matrix product involving the diagonal matrix, though the resulting Su^t is still transposed. See what is the best way to multiply a diagonal matrix in fortran

The following code can be compiled by g++ main.cpp -o main -llapacke -llapack -lgslcblas -lblas -lm -Wall (or -lcblas`...)

#include <iostream>
#include <string>
#include <fstream>  

#include <stdlib.h>
#include <stdio.h>
#include <math.h>



extern "C" { 
#include <lapacke.h>
#include <cblas.h>
}

int main(int argc, char *argv[])
{
const int n=2;

double a[n * n] = {2, -1, 2,1};
double b[n]={3,4};
double u[n * n], s[n],vt[n * n];

int lda = n, ldu = n, ldvt = n;

//computing the SVD
int info = LAPACKE_dgesdd(LAPACK_COL_MAJOR, 'A', n, n, a, lda, s,
               u, ldu, vt, ldvt);
if (info !=0){
std::cerr<<"Lapack error occured in dgesdd. error code :"<<info<<std::endl;
}


for (int i = 0; i < n; i++) {
        s[i] = 1.0 / s[i];       
}

const int aa = 1;
const int c = 0;

//double r1[n*n];
double r2[n*n];
double res[n];

//compute the  first multiplication s*u^T
// here : s is not a vector : it is a diagonal matrix. The ouput must be of size n*n
//cblas_dgemm( CblasColMajor,CblasNoTrans, CblasTrans, n, n, n, aa, u, ldvt, s, ldu, c, r1, n);
for (int i = 0; i < n; i++) {
cblas_dscal(n,s[i],&u[i*n],1);
}

//compute the second multiplication v^T^T=vs*u^T
cblas_dgemm( CblasColMajor,CblasTrans, CblasTrans, n, n, n, aa, vt, ldvt, u, ldu, c, r2, n);
//now, r2 is the pseudoinverse of a.
//now that we have the pseudoinverse A* solve by multiplying with b.
cblas_dgemm( CblasColMajor,CblasNoTrans, CblasNoTrans, n, 1, n, aa, r2, ldvt, b, ldu, c, res, n);


for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
    std::cout<<r2[i*n+j]<<" ";
}
}

std::cout<<std::endl;
}

It prints the expected result:

0.25 0.25 -0.5 0.5