SVD to solve harwell-boeing sparse a.x=b system in C/C++?

Question

Does anybody know of a sparse SVD solver for c++? My problem involves some badly conditioned matrices that might have zeroed columns/rows. My data is stored in a uBLAS matrix which is the Harwell-Boeing sparse format.

I am having some trouble finding:

The SVD solver

1. The SVD solver that can operate on sparse matrices. Lapack doesn't seem to be able to do this? I want to have sparse matrices passed to the function and sparse matrices output.
2. A way to recombine the results... So that I can read off the xs from x=b(A^-1). I would expect this to be x=(b)(v.(d^-1).(u^t))

I am hoping to recreate the following two steps from GSL

gsl_linalg_SV_decomp_jacobi (gsl_matrix * A, gsl_matrix * V, gsl_vector * S) 
gsl_linalg_SV_solve (const gsl_matrix * U, const gsl_matrix * V, const gsl_vector * S, const gsl_vector * b, gsl_vector * x)

I also have no clue how to wrap a FORTRAN library in c++. Where/Are there any PROPACK c/c++ bindings?

Edit 1: I'm having some trouble with PROPACK. Does PROPACK output sparse matrices? It seems to output V as "V(LDV,KMAX): DOUBLE PRECISION array." which would imply that it doesn't?

Answer 1

SVDLIBC is a C library with partial support for the Harwell-Boeing format. I am not familiar with the library, but on the surface it seems to match your requirements.

Answer 2

You mentioned PROPACK . Fortran is C-compatible, you just have to know how the calling convention works. I'm not certain, but I think that the function you want to call in PROPACK is dlansvd (assuming double-precision), which is documented as follows:

  subroutine dlansvd(jobu,jobv,m,n,k,kmax,aprod,U,ldu,Sigma,bnd,
 c     V,ldv,tolin,work,lwork,iwork,liwork,doption,ioption,info,
 c     dparm,iparm)


c     DLANSVD: Compute the leading singular triplets of a large and
c     sparse matrix by Lanczos bidiagonalization with partial
c     reorthogonalization.
c
c     Parameters:
c
c     JOBU: CHARACTER*1. If JOBU.EQ.'Y' then compute the left singular vectors.
c     JOBV: CHARACTER*1. If JOBV.EQ.'Y' then compute the right singular 
c           vectors.
c     M: INTEGER. Number of rows of A.
c     N: INTEGER. Number of columns of A.
c     K: INTEGER. Number of desired singular triplets. K <= MIN(KMAX,M,N)
c     KMAX: INTEGER. Maximal number of iterations = maximal dimension of
c           the generated Krylov subspace.
c     APROD: Subroutine defining the linear operator A. 
c            APROD should be of the form:
c
c           SUBROUTINE DAPROD(TRANSA,M,N,X,Y,DPARM,IPARM)
c           CHARACTER*1 TRANSA
c           INTEGER M,N,IPARM(*)
c           DOUBLE PRECISION X(*),Y(*),DPARM(*)
c
c           If TRANSA.EQ.'N' then the function should compute the matrix-vector
c           product Y = A * X.
c           If TRANSA.EQ.'T' then the function should compute the matrix-vector
c           product Y = A^T * X.
c           The arrays IPARM and DPARM are a means to pass user supplied
c           data to APROD without the use of common blocks.
c     U(LDU,KMAX+1): DOUBLE PRECISION array. On return the first K columns of U
c               will contain approximations to the left singular vectors 
c               corresponding to the K largest singular values of A.
c               On entry the first column of U contains the starting vector
c               for the Lanczos bidiagonalization. A random starting vector
c               is used if U is zero.
c     LDU: INTEGER. Leading dimension of the array U. LDU >= M.
c     SIGMA(K): DOUBLE PRECISION array. On return Sigma contains approximation
c               to the K largest singular values of A.
c     BND(K)  : DOUBLE PRECISION array. Error estimates on the computed 
c               singular values. The computed SIGMA(I) is within BND(I)
c               of a singular value of A.
c     V(LDV,KMAX): DOUBLE PRECISION array. On return the first K columns of V
c               will contain approximations to the right singular vectors 
c               corresponding to the K largest singular values of A.
c     LDV: INTEGER. Leading dimension of the array V. LDV >= N.
c     TOLIN: DOUBLE PRECISION. Desired relative accuracy of computed singular 
c            values. The error of SIGMA(I) is approximately 
c            MAX( 16*EPS*SIGMA(1), TOLIN*SIGMA(I) )
c     WORK(LWORK): DOUBLE PRECISION array. Workspace of dimension LWORK.
c     LWORK: INTEGER. Dimension of WORK.
c            If JOBU.EQ.'N' and JOBV.EQ.'N' then  LWORK should be at least
c            M + N + 9*KMAX + 2*KMAX**2 + 4 + MAX(M+N,4*KMAX+4).
c            If JOBU.EQ.'Y' or JOBV.EQ.'Y' then LWORK should be at least
c            M + N + 9*KMAX + 5*KMAX**2 + 4 + 
c            MAX(3*KMAX**2+4*KMAX+4, NB*MAX(M,N)), where NB>1 is a block 
c            size, which determines how large a fraction of the work in
c            setting up the singular vectors is done using fast BLAS-3 
c            operation. 
c     IWORK: INTEGER array. Integer workspace of dimension LIWORK.
c     LIWORK: INTEGER. Dimension of IWORK. Should be at least 8*KMAX if
c             JOBU.EQ.'Y' or JOBV.EQ.'Y' and at least 2*KMAX+1 otherwise.
c     DOPTION: DOUBLE PRECISION array. Parameters for LANBPRO.
c        doption(1) = delta. Level of orthogonality to maintain among
c          Lanczos vectors.
c        doption(2) = eta. During reorthogonalization, all vectors with
c          with components larger than eta along the latest Lanczos vector
c          will be purged.
c        doption(3) = anorm. Estimate of || A ||.
c     IOPTION: INTEGER array. Parameters for LANBPRO.
c        ioption(1) = CGS.  If CGS.EQ.1 then reorthogonalization is done
c          using iterated classical GRAM-SCHMIDT. IF CGS.EQ.0 then 
c          reorthogonalization is done using iterated modified Gram-Schmidt.
c        ioption(2) = ELR. If ELR.EQ.1 then extended local orthogonality is
c          enforced among u_{k}, u_{k+1} and v_{k} and v_{k+1} respectively.
c     INFO: INTEGER. 
c         INFO = 0  : The K largest singular triplets were computed succesfully
c         INFO = J>0, J<K: An invariant subspace of dimension J was found.
c         INFO = -1 : K singular triplets did not converge within KMAX
c                     iterations.   
c     DPARM: DOUBLE PRECISION array. Array used for passing data to the APROD
c         function.   
c     IPARM: INTEGER array. Array used for passing data to the APROD
c         function.   
c
c     (C) Rasmus Munk Larsen, Stanford, 1999, 2004 
c

In Fortran, the important things to remember is that all parameters are passed by reference , and non-sparse arrays are stored in column-major format. So, the proper declaration of this function in C++ should be as follows (untested):

extern "C"
void dlansvd(const char *jobu,
             const char *jobv,
             int *m,
             int *n,
             int *k,
             int *kmax,
             void (*aprod)(const char *transa,
                           int *m,
                           int *n,
                           int *iparm,
                           double *x,
                           double *y,
                           double *dparm),
             double *U,
             int *ldu,
             double *Sigma,
             double *bnd,
             double *V,
             int *ldv,
             double *tolin,
             double *work,
             int *lwork,
             int *iwork,
             int *liwork,
             double *doption,
             int *ioption,
             int *info,
             double *dparm,
             int *iparm);

It's quite a beast. Good luck!

Answer 3

It might be worthwhile checking out Tim Davis's sparse linear algebra software: http://www.cise.ufl.edu/~davis/

Generally speaking I've found his software to be really useful, typically very efficient and robust.

It seems that he's been working on a sparse SVD package with a student, but I'm not sure what stage the project is at.

Hope this helps.