Tutorials › Calculus II › Richardson Extrapolation for Derivative Estimates

Home  /  Tutorials  /  Taylor Methods in Computation

Taylor Methods in Computation · Tutorial 887 of 998

Richardson Extrapolation for Derivative Estimates

Combine finite-difference estimates at different step sizes to remove their leading error and improve a derivative approximation.

Advanced 9 min read

What You'll Learn

  • Identify the leading truncation-error term in a finite-difference estimate.
  • Derive Richardson extrapolation weights for an error of order \(h^p\).
  • Apply the centered-difference formula to improve a derivative estimate from second to fourth order.
  • Use extrapolation with a forward difference whose leading error is first order.
  • Check an extrapolated estimate against the exact derivative.
  • Recognize when roundoff or an invalid error model limits the method.

Why Combine Two Derivative Estimates?

A finite-difference formula estimates a derivative using function values at nearby points. Its accuracy depends on the step size \(h\): reducing \(h\) usually reduces truncation error, but making \(h\) too small can increase the effects of roundoff. The tutorial on truncation and roundoff in numerical differentiation discusses that tradeoff.

Richardson extrapolation takes a different step toward accuracy. Instead of relying only on a smaller step, it combines estimates at two step sizes so that their leading truncation errors cancel. The method is useful when the error has a predictable power of \(h\), and both estimates lie in a range where that error model is reasonable.

Central idea: If an estimate has the form \(D(h)=L+c h^p+\text{higher-order terms}\), then estimates at \(h\) and \(h/2\) can be weighted to eliminate the term \(c h^p\). Here \(L\) is the quantity being estimated, usually a derivative.

The Error Pattern in Finite Differences

For a differentiable function \(f\), the centered-difference estimate at \(x\) is

$$ D(h)=\frac{f(x+h)-f(x-h)}{2h}. $$

Taylor expansion about \(x\) gives, when \(f\) is sufficiently smooth,

$$ f(x+h)=f(x)+f'(x)h+\frac{f''(x)}{2}h^2+\frac{f'''(x)}{6}h^3+\cdots, $$

and the corresponding expansion for \(f(x-h)\) has alternating signs on the odd powers of \(h\). Subtracting the two expansions removes the even-power terms; dividing by \(2h\) gives

$$ D(h)=f'(x)+\frac{f'''(x)}{6}h^2+\frac{f^{(5)}(x)}{120}h^4+\cdots. $$

Thus, the centered difference has a leading error proportional to \(h^2\), provided the coefficient \(f'''(x)/6\) is not zero. The expansion also shows why a second estimate at a different step size can help: its leading error has the same coefficient but a predictable change in size.

Definition: Richardson extrapolation is a weighted combination of estimates computed at different step sizes, chosen to cancel one or more known leading terms in their error expansions.

Deriving the Extrapolation Weights

Suppose a family of estimates \(D(h)\) for a target value \(L\) has an expansion

$$ D(h)=L+c h^p+O(h^r), \qquad r>p, $$

where \(p>0\) is the known leading error power and \(c\) does not depend on \(h\). Replacing \(h\) by \(h/2\) gives

$$ D(h/2)=L+\frac{c h^p}{2^p}+O(h^r). $$

Seek a weighted combination \(A D(h/2)+B D(h)\) that preserves \(L\) and cancels \(c h^p\). The weights must satisfy \(A+B=1\) and \(A/2^p+B=0\). Solving these equations gives \(A=2^p/(2^p-1)\) and \(B=-1/(2^p-1)\).

$$ D_{\mathrm{R}}(h) = \frac{2^pD(h/2)-D(h)}{2^p-1}. $$

The cancellation can also be checked directly: the coefficient of \(c h^p\) in the numerator is \(2^p/2^p-1=0\), while the coefficient of \(L\) is \(2^p-1\). After division by \(2^p-1\), the result estimates \(L\) with the leading error removed. If the remaining terms really are of order \(h^r\) or smaller, the extrapolated estimate has error \(O(h^r)\).

For the centered difference, \(p=2\). The resulting formula is

$$ D_{\mathrm{R}}(h)=\frac{4D(h/2)-D(h)}{3}. $$

Under a sufficiently smooth error expansion for the centered difference, the \(h^2\) term cancels and the remaining leading error is generally of order \(h^4\). The method does not guarantee that the error is exactly a constant times \(h^4\); that conclusion depends on the smoothness and the validity of the expansion over the step sizes used.

Worked Example: Centered Difference for a Polynomial

Worked Example: Centered Difference for a Polynomial

Estimate \(f'(1)\) for \(f(x)=x^5\) using the centered difference with \(h=0.2\) and \(h/2=0.1\), then extrapolate.

First calculate the estimate at \(h=0.2\):

$$ D(0.2) = \frac{(1.2)^5-(0.8)^5}{0.4} = \frac{2.48832-0.32768}{0.4} = \frac{2.16064}{0.4} = 5.4016. $$

At the half-step \(h/2=0.1\):

$$ D(0.1) = \frac{(1.1)^5-(0.9)^5}{0.2} = \frac{1.61051-0.59049}{0.2} = \frac{1.02002}{0.2} = 5.1001. $$

Use the centered-difference Richardson formula:

$$ D_{\mathrm{R}}(0.2) = \frac{4(5.1001)-5.4016}{3} = \frac{20.4004-5.4016}{3} = \frac{14.9988}{3} = 4.9996. $$

The exact derivative is \(f'(x)=5x^4\), so \(f'(1)=5\). The extrapolated estimate differs from the exact value by \(4.9996-5=-0.0004\), while the original estimates differ by \(0.4016\) and \(0.1001\). The expansion verifies the remaining error: direct binomial expansion gives \(D(h)=5+10h^2+h^4\), so the extrapolated value is \(5-h^4/4\). At \(h=0.2\), this is \(5-0.0016/4=4.9996\), as calculated.

Worked Example: Centered Difference for Sine

Worked Example: Centered Difference for Sine

Estimate \(f'(0)\) for \(f(x)=\sin x\) using \(h=0.4\) and \(h/2=0.2\). Round the displayed numerical values to nine decimal places.

Since \(\sin(-h)=-\sin h\), the centered difference at zero is \(D(h)=\sin(h)/h\). Using \(\sin(0.4)\approx0.389418342\) and \(\sin(0.2)\approx0.198669331\), the two estimates are

$$ D(0.4)=\frac{0.389418342}{0.4}\approx0.973545855, \qquad D(0.2)=\frac{0.198669331}{0.2}\approx0.993346655. $$

Apply the extrapolation weights:

$$ D_{\mathrm{R}}(0.4) = \frac{4(0.993346655)-0.973545855}{3} = \frac{3.973386620-0.973545855}{3} = \frac{2.999840765}{3} \approx0.999946922. $$

Differentiating \(f(x)=\sin x\) gives \(f'(x)=\cos x\), so \(f'(0)=1\). The estimate is about \(0.000053078\) below the exact value. This is consistent with cancellation of the leading \(h^2\) error. The numerical values above have been rounded for display, so the final digits of the computed estimate are affected by that rounding.

Using a Different Leading Error Power

Richardson extrapolation is not restricted to centered differences. A forward difference

$$ F(h)=\frac{f(x+h)-f(x)}{h} $$

has the Taylor expansion \(F(h)=f'(x)+\frac{f''(x)}{2}h+O(h^2)\) when \(f\) is sufficiently smooth. Its leading error has power \(p=1\), so the appropriate weights are

$$ F_{\mathrm{R}}(h)=2F(h/2)-F(h). $$

The factor \(2\) is specific to a first-order leading error. Using the centered-difference weights \(4/3\) and \(-1/3\) here would not cancel the forward difference's \(h\) term.

Worked Example: Forward Difference for the Natural Logarithm

Estimate \(f'(1)\) for \(f(x)=\ln x\) using forward differences with \(h=0.2\) and \(h/2=0.1\). Round the displayed logarithms and final estimate to nine decimal places.

The estimates are

$$ F(0.2) = \frac{\ln(1.2)-\ln(1)}{0.2} = \frac{0.182321557-0}{0.2} \approx0.911607785, $$

and

$$ F(0.1) = \frac{\ln(1.1)-\ln(1)}{0.1} = \frac{0.095310180-0}{0.1} \approx0.953101800. $$

Because the leading error is first order, use \(F_{\mathrm{R}}(h)=2F(h/2)-F(h)\):

$$ F_{\mathrm{R}}(0.2) = 2(0.953101800)-0.911607785 = 1.906203600-0.911607785 \approx0.994595815. $$

The exact derivative is \(f'(x)=1/x\), so \(f'(1)=1\). The extrapolated estimate is about \(0.005404185\) below the exact value. Its remaining error is consistent with the \(O(h^2)\) term left after the leading first-order error is canceled. The approximation is improved here, but extrapolation does not promise that every particular pair of step sizes will yield a smaller numerical error.

When Extrapolation Is Useful—and When to Be Careful

The method is most effective when three conditions hold: the estimates follow a reliable expansion in powers of \(h\); the leading power \(p\) is known; and both step sizes are small enough for the expansion to describe their errors. Smoothness of the function near the evaluation point supports the Taylor expansions used above. If the function is not sufficiently smooth, or a step crosses a point where its behavior changes, the assumed error pattern may fail.

The formula also combines errors already present in the two estimates. For example, the centered formula uses weights \(4/3\) and \(-1/3\). If function values contain roundoff or measurement noise, the combination can amplify some of that error. Making \(h\) smaller is therefore not automatically better: truncation error may decrease while roundoff effects become more significant.

Practical check: Compare estimates at successive step sizes and confirm that their changes are consistent with the expected leading error power before trusting the extrapolated result. Such agreement is evidence for the error model, not a proof that it holds exactly.

A common mistake is to use weights for the wrong error power. For example, applying the centered-difference formula to a forward difference does not cancel its leading error. If \(F(h)=L+c h+O(h^2)\), then

$$ \frac{4F(h/2)-F(h)}{3} = \frac{4(L+c h/2+O(h^2))-(L+c h+O(h^2))}{3} = L+\frac{c h}{3}+O(h^2). $$

The \(h\) term remains, so this combination has not achieved the intended cancellation. The correction is to use the weights derived for \(p=1\): \(2F(h/2)-F(h)\). Matching the weights to the actual leading error term is essential.

Summary

Richardson extrapolation improves a derivative estimate by combining two step sizes with weights selected from the leading error power. If \(D(h)=L+c h^p+\) higher-order terms, then

$$ D_{\mathrm{R}}(h)=\frac{2^pD(h/2)-D(h)}{2^p-1} $$

cancels the \(h^p\) term. In particular, centered differences have a leading \(h^2\) error and use \((4D(h/2)-D(h))/3\); forward differences have a leading \(h\) error and use \(2F(h/2)-F(h)\). The weights depend on the error model, and the method is reliable only when that model and the smoothness assumptions are appropriate.

Check Your Understanding

Work through these questions using the error expansion as well as the numerical formulas.

  1. If \(D(h)=L+c h^3+O(h^4)\), what extrapolation weights cancel the leading error when the step is halved?
  2. Why does the centered-difference formula use different weights from the forward-difference formula?
  3. For \(D(h)=L+c h^2+d h^4+O(h^6)\), what order of error remains after centered-difference extrapolation?
  4. What assumption about the function and the step sizes supports using a Taylor-based error expansion?
  5. Why can making the step size extremely small reduce the practical benefit of extrapolation?

Previous

Next

Back to Tutorials