Runge-Kutta 4 in python

Question

I have a question, in the code, for h=0.1 shows a minor error that h=0.01 and h=0.001. I don't understand why?, but with h=0.0001 the error again decreases.

thanks!

def f(x,y):
    return 2*x**2-4*x+y

def RK4(x0,y0):
    while x0 < b:
        k1 = h*f(x0,y0)
        k2 = h*f(x0+0.5*h,y0+0.5*k1)
        k3 = h*f(x0+0.5*h,y0+0.5*k2)
        k4 = h*f(x0+h,y0+k3)
        y0+=(k1+2*k2+2*k3+k4)/6
        x0+=h
    return y0

b=3
h=0.001
print(RK4(1,0.7182818))

Answer 1

Error analysis

If you print also the last x0 , then you will see the iteration never stops at exactly b . The floating point representation of h will be off by some fraction of the machine epsilon. If it is slightly larger, then the iteration will perform the correct number of loops. If it is smaller, then the iteration will perform one loop more than necessary and stop at a little less than b+h .

Additionally, the linear DE of this test problem has an easily computed exact solution

y' - y = f'(x) - f(x),  f(x) = -2*x^2
=> (y(x)-f(x))*exp(-x) = (y0-f(x0))*exp(-x0)

so that the flow function of the solution is

def phi(x, x0,y0): return (y0+2*x0**2)*np.exp(x-x0)-2*x**2

giving for the original code the results

exact solution: 2.085536712902183

 h           returned   x       returned   y      to exact at b  to exact at ret. x
------------------------------------------------------------------------------------
   0.1 : 3.00000000000000178  2.08553122271193736  -5.49019e-06 -5.49019e-06
  0.01 : 3.00999999999997936  2.16719971215161866      0.081663 -6.90252e-10
 0.001 : 3.00099999999977962  2.09363029572970216    0.00809358 -3.81029e-13
0.0001 : 3.00000000000200018  2.08553671291035991   8.17701e-12 -7.99849e-12
 1e-05 : 3.00000000001310241  2.08553671302741339    1.2523e-10  1.92886e-11

In the overshooting cases the error is visibly a fraction of h , while the other lines show the expected order 4 convergence competing with the floating point error accumulation.

Correction variants

You can correct this by a-priori computing the number of steps or by correcting the last step

def f(x,y):
    return 2*x**2-4*x+y

def RK4(x0,y0,xf,h):
    while x0 < xf:
        if x0+h > xf: h=xf-x0
        k1 = h*f(x0,y0)
        k2 = h*f(x0+0.5*h,y0+0.5*k1)
        k3 = h*f(x0+0.5*h,y0+0.5*k2)
        k4 = h*f(x0+h,y0+k3)
        y0+=(k1+2*k2+2*k3+k4)/6
        x0+=h
    return x0,y0

b=3
for k in range (1,5): 
    h0=10**-k; 
    for h in [2*h0, h0, 0.5*h0]:
        xf,yf = RK4(x0, y0,b,h); 
        print(f'{h:6} : {xf:.17f}  {yf:.17f}  {yf-phi(b,x0,y0):12.6g}  {(yf-phi(b,x0,y0))/h**4:12.6g}')

which gives the desired results

 h          returned  x               y               error          error/h^4
--------------------------------------------------------------------------------
   0.2 : 3.00000000000000000  2.08546790477393795  -6.88081e-05    -0.0430051
   0.1 : 3.00000000000000000  2.08553122271192359  -5.49019e-06    -0.0549019
  0.05 : 3.00000000000000000  2.08553632856235494   -3.8434e-07    -0.0614944
  0.02 : 3.00000000000000000  2.08553670239502909  -1.05072e-08    -0.0656697
  0.01 : 3.00000000000000000  2.08553671223127068  -6.70912e-10    -0.0670912
 0.005 : 3.00000000000000000  2.08553671285973685   -4.2446e-11    -0.0679137
 0.002 : 3.00000000000000000  2.08553671290148124  -7.01661e-13    -0.0438538
 0.001 : 3.00000000000000000  2.08553671290180187  -3.81029e-13     -0.381029
0.0005 : 3.00000000000000000  2.08553671290032216  -1.86073e-12      -29.7717
0.0002 : 3.00000000000000000  2.08553671290186093  -3.21965e-13      -201.228
0.0001 : 3.00000000000000000  2.08553671289418929  -7.99361e-12      -79936.1
 5e-05 : 3.00000000000000000  2.08553671292377762   2.15947e-11   3.45516e+06

The computed error coefficient shows that the order 4 method error dominates for h between 0.005 and 0.1 , for larger step sizes the higher order error terms are too large, for lower h the number of necessary steps has increased so much that the accumulation of floating point errors dominates the method error.

---

As announced, you can pre-compute the number N of steps and ensure that N*h=xf-x0 . For that replace

    while x0 < xf:
        if x0+h > xf: h=xf-x0

with

    Dx = float(xf-x0); N = int(0.5+Dx/h); h = Dx/N
    for _ in range(N):

where you can still observe the accumulation of floating point errors in x0 ,

0.1     (3.0000000000000018, 2.0855312227119374)
0.01    (2.9999999999999796, 2.0855367122311055)
0.001   (2.9999999999997797, 2.085536712900021 )
0.0001  (3.000000000002,     2.08553671291036  )
1e-05   (3.0000000000131024, 2.0855367130274134)