C++ 的數值導數

Question

這個問題在不同的平台上出現了一千次。 但是，我仍然需要了解一些事情。 這是一個完整的例子：

#include <iostream>
#include <iomanip>
#include <cmath>

// function to derivative
double f (double var, void * params){
  (void)(params); 
  return pow (var, 1.5);
}

// Naive central step derivative
double derivative1(double var, double f(double,void*), double h){ 
return (f(var+h,NULL) - f(var-h,NULL) )/2.0/h; 
}

// Richardson 5-point rule
double gderivative(double var, double f(double,void*), double h0){
 return(4.0*derivative1(var,f,h0) - derivative1(var,f,2.0*h0))/3.0;
}


int main (void){


  for (int i=10;i>0;i--){
  double h0=pow(10,-i);
  double x=2.0;
  double exact = 1.5 * sqrt(x);
  double test1=derivative1(x,f,h0);
  double gtest=gderivative(x,f,h0);
  std::cout << "h0 = " << h0 << std::endl;
  std::cout << "Exact      = "  << std::scientific<<std::setprecision(15) << exact << std::endl;
  std::cout << "Naive step = "  << std::setprecision(15) << test1 <<", diff = " << std::setprecision(15)<< exact-test1 << ", percent error = " << (exact-test1)/exact*100.0 << std::endl;
  std::cout << "Richardson = "  << std::setprecision(15) << gtest <<", diff = " << std::setprecision(15)<< exact-gtest << ", percent error = " << (exact-gtest)/exact*100.0 << std::endl;



}


 return 0;
}

output 是

h0 = 1e-10
Exact      = 2.121320343559643e+00
Naive step = 2.121318676273631e+00, diff = 1.667286011475255e-06, percent error = 7.859661632610832e-05
Richardson = 2.121318306199290e+00, diff = 2.037360352868944e-06, percent error = 9.604208808228318e-05
h0 = 1.000000000000000e-09
Exact      = 2.121320343559643e+00
Naive step = 2.121320674675076e+00, diff = -3.311154328500265e-07, percent error = -1.560893119491818e-05
Richardson = 2.121320748689944e+00, diff = -4.051303013064000e-07, percent error = -1.909802555452698e-05
h0 = 1.000000000000000e-08
Exact      = 2.121320343559643e+00
Naive step = 2.121320341608168e+00, diff = 1.951474537520426e-09, percent error = 9.199339191957163e-08
Richardson = 2.121320341608168e+00, diff = 1.951474537520426e-09, percent error = 9.199339191957163e-08
h0 = 1.000000000000000e-07
Exact      = 2.121320343559643e+00
Naive step = 2.121320341608168e+00, diff = 1.951474537520426e-09, percent error = 9.199339191957163e-08
Richardson = 2.121320340868019e+00, diff = 2.691623368633600e-09, percent error = 1.268843424240664e-07
h0 = 1.000000000000000e-06
Exact      = 2.121320343559643e+00
Naive step = 2.121320343606570e+00, diff = -4.692690680485612e-11, percent error = -2.212155601454860e-09
Richardson = 2.121320343643577e+00, diff = -8.393419292929138e-11, percent error = -3.956695799581460e-09
h0 = 1.000000000000000e-05
Exact      = 2.121320343559643e+00
Naive step = 2.121320343584365e+00, diff = -2.472244631235299e-11, percent error = -1.165427295665677e-09
Richardson = 2.121320343595468e+00, diff = -3.582467655860455e-11, percent error = -1.688791448560268e-09
h0 = 1.000000000000000e-04
Exact      = 2.121320343559643e+00
Naive step = 2.121320343340116e+00, diff = 2.195266191051815e-10, percent error = 1.034858406801534e-08
Richardson = 2.121320343561791e+00, diff = -2.148059508044753e-12, percent error = -1.012604963020456e-10
h0 = 1.000000000000000e-03
Exact      = 2.121320343559643e+00
Naive step = 2.121320321462283e+00, diff = 2.209735949776359e-08, percent error = 1.041679516479040e-06
Richardson = 2.121320343559311e+00, diff = 3.317346397579968e-13, percent error = 1.563812088849040e-11
h0 = 1.000000000000000e-02
Exact      = 2.121320343559643e+00
Naive step = 2.121318133840577e+00, diff = 2.209719065504601e-06, percent error = 1.041671557157002e-04
Richardson = 2.121320343601055e+00, diff = -4.141265108614789e-11, percent error = -1.952211093995174e-09
h0 = 1.000000000000000e-01
Exact      = 2.121320343559643e+00
Naive step = 2.121099269013200e+00, diff = 2.210745464426012e-04, percent error = 1.042155406248691e-02
Richardson = 2.121320759832334e+00, diff = -4.162726914280768e-07, percent error = -1.962328286210455e-05

我相信標准GSL gsl_deriv_central采用了理查森程序。

現在給出的選擇h0的共同論點是，理論上它應該選擇盡可能小以提高導數的精度，但在數值上它不應該太小，這樣我們會達到浮點舍入從而破壞精度。 經常有人說最佳選擇應該在1e-6 - 1e-8左右。 我的問題是：

泛型導數中h0的最佳選擇是什么？

我應該逐案檢查嗎？ 通常可能無法得到准確的結果來檢查。 在這種情況下應該怎么做？ 現在在這種特殊情況下， Naive step的最佳選擇似乎是h0 = 1.000000000000000e-05而對於Richardson h0 = 1.000000000000000e-03 。 這讓我很困惑，因為這些都不小。 對任何其他既有效又精確（ double ）的好選擇（簡單算法/庫）有什么建議嗎？

Answer 1

這完全符合預期。 中心差對精確導數有二階誤差O(h^2) 。 function 評估的誤差為mu量級，即機器常數（用於適度縮放的測試示例）。 因此，中心差的評估誤差的量級為mu/h 。 如果這兩個影響大致相等，則總體誤差最小，因此h=mu^(1/3)給出h=1e-5 ，誤差約為1e-10 。

Richardson 外推的相同計算給出了精確導數的誤差階O(h^4) ，導致h=mu^(1/5)=1e-3作為最佳步長，誤差約為1e-12 。

loglog plot 在更大的步長樣本上兩種方法的誤差

在實際應用中，您可能希望縮放h以使指示的大小相對於x的大小。 確切的最佳值還取決於 function 的衍生物的大小，如果它們增長得太快的話。

要獲得更精確的導數值，您需要更高或多精度的數據類型，或使用自動/算法微分，其中可以以與 function 值相同的精度評估一階和二階導數。

Answer 2

個人意見警告

根據我的經驗，我發現使用與其影響的變量相比較小的步長會更好。

例如，我通常使用這樣的東西：

auto dfdx(std::function<double (double)> f, double x, double h0=1e-5){
    h0=std::abs(x)*h0+h0; // this is 1e-5 if x<<1e-5, and |x|*1e-5 otherwise
    
    return ( f(x+h0) - f(x-h0) )/2/h0;
}

這應該可行，因為有限差分是由泰勒展開驅動的。 也就是說，只要 x<<h0，有限差分應該是一個很好的近似。