Why accurate Taylor approximations can give inaccurate computations
A Taylor polynomial may approximate a function extremely well and still produce a poor computed answer. The issue is that evaluation requires arithmetic on the polynomial’s terms, and those operations are performed with finite precision. If large terms nearly cancel, small rounding errors in them can become large relative to the small result.
It is useful to distinguish two sources of error. Truncation error comes from replacing a function by a Taylor polynomial of finite degree. Roundoff error comes from evaluating that polynomial using finite-precision arithmetic. Improving one does not necessarily improve the other: increasing the degree addresses truncation, while extra precision or a more suitable formula addresses roundoff.
Cancellation and sensitivity
Suppose a quantity is written as a sum \(s=a_1+\cdots+a_m\). Cancellation occurs when positive and negative terms of substantial size combine to give a much smaller result. A useful measure of the sum’s sensitivity to small relative changes in its terms is its cancellation condition number:
If the terms are all of the same sign, this ratio is \(1\). If the terms nearly cancel, the denominator is small and the ratio can be large. A large value warns that even small relative errors in the terms may cause a large relative error in the sum.
In a standard floating-point model, an individual arithmetic operation is often described by \(\operatorname{fl}(a\mathbin{\circ}b)=(a\mathbin{\circ}b)(1+\delta)\), where \(\circ\) is an operation and \(|\delta|\le u\). Here \(u\) is the unit roundoff, and the model assumes the operation does not overflow or enter a range where underflow makes the model inappropriate. This is an approximation to how floating-point arithmetic behaves, not a guarantee that every calculation has the same error.
How cancellation amplifies small errors
Consider terms \(a_i\) that are each perturbed by a small relative amount \(\delta_i\), with \(|\delta_i|\le u\). The perturbed sum differs from the exact sum by \(\sum_i a_i\delta_i\). The triangle inequality gives a bound on that difference:
When the exact sum \(s=\sum_i a_i\) is nonzero, dividing by \(|s|\) gives the relative-error estimate
This derivation explains the role of cancellation: the same scale of relative error in the individual terms can have a much bigger relative effect on their sum when \(\kappa_{\mathrm{sum}}\) is large. In an actual calculation, intermediate rounding in multiplications and additions can add further error, so this estimate is a diagnostic rather than a complete error bound for every evaluation procedure.
Worked Example: A Taylor polynomial with nearly cancelling terms
Worked Example: A quadratic Taylor polynomial near a root
Let \(f(x)=(1-x)^2\). Its Taylor polynomial about \(x=0\) is exactly \(P_2(x)=1-2x+x^2\): expanding the function gives \(1-2x+x^2\), and there are no higher-degree terms. Evaluate it at \(x=0.9999999\), first using seven-significant-digit decimal arithmetic, rounding each operation to seven significant digits.
The exact function value is
Now evaluate the expanded polynomial. First compute its terms, rounding the product \(2x\) and the product \(x^2\) to seven significant digits:
Combining the terms from left to right with the same rounding rule gives
The computed result is \(-2\times10^{-7}\), although the true result is \(10^{-14}\). The absolute error is \(2.0000001\times10^{-7}\), and the relative error is approximately \(2.0000001\times10^7\). The polynomial itself is exact, so its truncation error is zero; the failure here is due to finite-precision evaluation.
The terms are poorly conditioned as a sum. Their cancellation condition number is
In contrast, evaluate the equivalent factored expression using the same seven-significant-digit arithmetic:
In this particular decimal-arithmetic example, the subtraction and square retain the needed result. Substitution into the original function confirms that the factored value is correct: \(f(0.9999999)=10^{-14}\). Equivalent formulas need not behave equally well in finite precision.
Separating truncation error from roundoff error
Let \(P_n(x)\) be the degree-\(n\) Taylor polynomial used to approximate \(f(x)\), and let \(\widehat{P}_n(x)\) denote the result actually computed in finite precision. Then the total error can be decomposed exactly:
The first parenthesis is the polynomial’s truncation error; the second is the evaluation error, which includes roundoff. A Taylor remainder estimate bounds the first quantity. It cannot establish that the second is small. This is why a degree chosen to meet an approximation tolerance may still be unsuitable if the resulting polynomial is evaluated in a cancellation-prone way.
Worked Example: Subtracting nearly equal quantities
Worked Example: Evaluating \(e^x-1\) near zero
Consider \(e^x-1\) at \(x=10^{-10}\). The degree-two Taylor polynomial for \(e^x\) at zero is \(P_2(x)=1+x+x^2/2\). Suppose calculations use decimal arithmetic with eight significant digits and round after each operation.
Evaluating \(P_2(x)\) begins with
The quadratic term is
Adding that term to \(1.0000000\) still rounds to \(1.0000000\). Subtracting \(1\) therefore produces the computed result \(0\). The subtraction loses the small difference that was the quantity of interest.
Instead, use the algebraically equivalent Taylor polynomial for \(e^x-1\), obtained by subtracting the constant term before evaluation:
Rounded to eight significant digits, this is \(1.0000000\times10^{-10}\), rather than zero. For a check against the function, Taylor’s theorem gives a remainder after degree two of \(e^\xi x^3/6\) for some \(\xi\) between \(0\) and \(10^{-10}\). Its magnitude is less than \(2\times10^{-31}\), since \(e^\xi<2\) on this interval. Thus the polynomial approximation has a tiny truncation error, while the direct finite-precision calculation still returned zero. The reformulation avoids the damaging final subtraction.
Worked Example: When more precision does not fix truncation
Worked Example: Approximating \(e^{0.1}\) with degree two
Use \(P_2(x)=1+x+x^2/2\) to approximate \(e^{0.1}\). The polynomial value is
The function value is approximately \(e^{0.1}=1.105170185\ldots\), so the actual error is approximately
Taylor’s theorem gives the remainder \(R_2=e^\xi(0.1)^3/6\) for some \(0<\xi<0.1\). Since \(e^\xi<e^{0.1}<1.106\),
The actual error is within this bound. If the polynomial is evaluated accurately, increasing arithmetic precision will leave the approximation near \(1.105\); the remaining difference from \(e^{0.1}\) is truncation error. To reduce that error, use a higher-degree Taylor polynomial or another approximation with a suitable remainder bound. Substitution into \(P_2\) verifies the computed polynomial value: \(1+0.1+0.005=1.105\).
Choosing a reliable evaluation strategy
When a Taylor calculation produces a result much smaller than its individual terms, pause before treating the displayed digits as trustworthy. The following checks help identify the source of trouble:
Use a valid Taylor remainder bound to determine how close the polynomial itself is to the function.
If large positive and negative terms produce a small sum, the sum’s cancellation condition number may be large.
Compare the computed value with an algebraically equivalent form that avoids subtracting nearly equal quantities, when such a form is available.
If the result changes substantially as precision increases, roundoff is significant. If it stabilizes but remains different from the function, truncation may dominate.
Neither reformulation nor extra precision is a substitute for checking the approximation error. A stable evaluation of a low-degree polynomial can still be a poor approximation to the function. Likewise, a very accurate Taylor approximation can be spoiled by an unstable evaluation formula.
Check Your Understanding
For each question, identify what the given information does—and does not—tell you about the computed result.
- What does a large cancellation condition number indicate about a sum?
- Why does a small Taylor remainder bound not guarantee that a computed polynomial value is accurate?
- For \(f(x)=(1-x)^2\), why can \(1-2x+x^2\) be less reliable near \(x=1\) than \((1-x)^2\)?
- In the calculation of \(e^x-1\) for very small \(x\), why can evaluating \(e^x\) and then subtracting \(1\) lose the desired result?
- If a computed answer changes when arithmetic precision is increased, which error source should you investigate? If it remains stable but disagrees with the function, what should you investigate?
