Why ceres covariance.Compute() seems run forever and not return?

Question

I am trying to write a BA optimizer using Ceres and want to compute the covariances for my optimized results. But the program stuck at covariance.Compute(covariance_blocks, &problem) and it seems never stop computing and runs forever. I debugged deep inside the covariance.Compute() function and noticed that it stuck at the Eigen::SparseQR solver. The optimization steps work fine. Here is my full report from ceres.

Solver Summary (v 1.14.0-eigen-(3.3.5)-no_lapack-eigensparse-openmp-no_tbb)

                                     Original                  Reduced
Parameter blocks                         1335                     1335
Parameters                              20025                    20025
Residual blocks                          1780                     1780
Residuals                               22677                    22677

Minimizer                        TRUST_REGION

Sparse linear algebra library    EIGEN_SPARSE
Trust region strategy     LEVENBERG_MARQUARDT

                                        Given                     Used
Linear solver          SPARSE_NORMAL_CHOLESKY   SPARSE_NORMAL_CHOLESKY
Threads                                     1                        1
Linear solver ordering              AUTOMATIC                     1335

Cost:
Initial                          2.021075e+09
Final                            5.734662e+03
Change                           2.021070e+09

Minimizer iterations                       25
Successful steps                           25
Unsuccessful steps                          0

Time (in seconds):
Preprocessor                         0.000846

  Residual only evaluation           0.021398 (25)
  Jacobian & residual evaluation     0.699060 (25)
  Linear solver                      0.557998 (25)
Minimizer                            1.327035

Postprocessor                        0.000104
Total                                1.327986

Termination:                      CONVERGENCE (Function tolerance reached. |cost_change|/cost: 2.763070e-10 <= 1.000000e-06)

iter      cost      cost_change  |gradient|   |step|    tr_ratio  tr_radius  ls_iter  iter_time  total_time
   0  3.179329e+07    0.00e+00    3.91e+10   0.00e+00   0.00e+00  1.00e+04        0    1.08e-01    1.20e-01
   1  1.517378e+06    3.03e+07    7.50e+06   9.20e+01   1.00e+00  3.00e+04        1    8.24e-01    9.44e-01
   2  1.578857e+05    1.36e+06    1.26e+06   3.50e+01   1.00e+00  9.00e+04        1    7.31e-01    1.68e+00
   3  7.079412e+04    8.71e+04    2.09e+05   1.59e+01   1.00e+00  2.70e+05        1    7.24e-01    2.40e+00
   4  6.395899e+04    6.84e+03    7.87e+04   9.33e+00   1.02e+00  8.10e+05        1    7.21e-01    3.12e+00
   5  5.746863e+04    6.49e+03    7.13e+04   4.92e+00   1.02e+00  2.43e+06        1    7.25e-01    3.84e+00
   6  4.865750e+04    8.81e+03    8.72e+04   3.41e+00   1.01e+00  7.29e+06        1    7.23e-01    4.57e+00
   7  4.089894e+04    7.76e+03    9.71e+04   6.98e+00   1.02e+00  2.19e+07        1    7.22e-01    5.29e+00
   8  3.531157e+04    5.59e+03    1.07e+05   1.63e+01   1.05e+00  6.56e+07        1    7.25e-01    6.02e+00
   9  2.937695e+04    5.93e+03    1.88e+05   2.85e+01   1.04e+00  1.97e+08        1    7.28e-01    6.74e+00
  10  2.435229e+04    5.02e+03    3.88e+05   3.19e+01   9.59e-01  5.90e+08        1    7.23e-01    7.47e+00
  11  2.065070e+04    3.70e+03    2.95e+05   2.39e+01   1.04e+00  1.77e+09        1    7.22e-01    8.19e+00
  12  1.886882e+04    1.78e+03    9.54e+04   1.43e+01   1.13e+00  5.31e+09        1    7.23e-01    8.91e+00
  13  1.828538e+04    5.83e+02    1.20e+05   1.16e+01   1.08e+00  1.59e+10        1    7.24e-01    9.64e+00
  14  1.790181e+04    3.84e+02    7.20e+04   1.79e+01   1.04e+00  4.78e+10        1    7.19e-01    1.04e+01
  15  1.759101e+04    3.11e+02    1.18e+05   2.76e+01   1.03e+00  1.43e+11        1    7.20e-01    1.11e+01
  16  1.739361e+04    1.97e+02    2.49e+05   3.36e+01   1.03e+00  4.30e+11        1    7.21e-01    1.18e+01
  17  1.733176e+04    6.19e+01    9.70e+04   2.12e+01   1.10e+00  1.29e+12        1    7.22e-01    1.25e+01
  18  1.732284e+04    8.92e+00    1.19e+04   6.60e+00   1.15e+00  3.87e+12        1    7.21e-01    1.32e+01
  19  1.732184e+04    9.95e-01    3.75e+03   1.57e+00   1.26e+00  1.16e+13        1    7.29e-01    1.40e+01
  20  1.732163e+04    2.10e-01    2.29e+03   9.22e-01   1.62e+00  3.49e+13        1    7.26e-01    1.47e+01
  21  1.732153e+04    1.04e-01    1.51e+03   4.64e-01   1.75e+00  1.05e+14        1    7.23e-01    1.54e+01
  22  1.732147e+04    6.23e-02    1.06e+03   9.74e-02   1.80e+00  3.14e+14        1    7.27e-01    1.61e+01
  23  1.732143e+04    4.05e-02    7.78e+02   2.55e-02   1.82e+00  9.41e+14        1    7.24e-01    1.69e+01
  24  1.732140e+04    2.78e-02    5.92e+02   1.87e-02   1.84e+00  2.82e+15        1    7.19e-01    1.76e+01
  25  1.732138e+04    1.97e-02    4.56e+02   1.48e-02   1.85e+00  8.47e+15        1    7.17e-01    1.83e+01

Solver Summary (v 1.14.0-eigen-(3.3.5)-no_lapack-eigensparse-openmp-no_tbb)

                                     Original                  Reduced
Parameter blocks                         2552                     2552
Parameters                              23679                    23679
Residual blocks                         26098                    26098
Residuals                               71313                    71313

Minimizer                        TRUST_REGION

Sparse linear algebra library    EIGEN_SPARSE
Trust region strategy     LEVENBERG_MARQUARDT

                                        Given                     Used
Linear solver          SPARSE_NORMAL_CHOLESKY   SPARSE_NORMAL_CHOLESKY
Threads                                     1                        1
Linear solver ordering              AUTOMATIC                     2552

Cost:
Initial                          3.179329e+07
Final                            1.732138e+04
Change                           3.177597e+07

Minimizer iterations                       26
Successful steps                           26
Unsuccessful steps                          0

Time (in seconds):
Preprocessor                         0.012063

  Residual only evaluation           0.135580 (26)
  Jacobian & residual evaluation     3.066419 (26)
  Linear solver                     15.525091 (26)
Minimizer                           18.891099

Postprocessor                        0.001107
Total                               18.904269

Termination:                      CONVERGENCE (Function tolerance reached. |cost_change|/cost: 7.930327e-07 <= 1.000000e-06)

The codes for setting up covariance are

    ceres::Covariance::Options options_cov;

    ceres::Covariance covariance(options_cov);
    std::vector<std::pair<const double*, const double*> > covariance_blocks;

    covariance_blocks.push_back(std::make_pair(cameraIntrinsic, cameraIntrinsic));
    covariance_blocks.push_back(std::make_pair(delta_theta_ci.data(), delta_theta_ci.data()));
    covariance_blocks.push_back(std::make_pair(cameraIntrinsic, delta_theta_ci.data()));

the cameraIntrinsic and delta_theta_ci are two arrays that I want to compute covariance. Can anyone help me?

Answer 1

Alright, I found the problem myself. The problem is exactly on the Eigen_Sparse QR solver. There is nothing wrong with the optimization problem I build. If I switched to another linear algebra library like options_cov.sparse_linear_algebra_library_type = ceres::SparseLinearAlgebraLibraryType::SUITE_SPARSE; The covariance can be computed.