Deriving Finite-Difference Formulas by Taylor Matching

From Taylor Expansions to Difference Formulas

A derivative describes the rate at which a function changes at a point. In computation, we may know function values at nearby points but not have a convenient way to evaluate the derivative directly. A finite-difference formula estimates the derivative by combining those sampled values.

The coefficients in such a formula are not arbitrary. Taylor expansions express nearby function values in terms of derivatives at the target point. By choosing coefficients that cancel unwanted Taylor terms, we can isolate the first derivative. This coefficient-matching process derives the forward, backward, and centered formulas and also reveals what assumptions justify their accuracy.

Taylor Matching as a System of Equations

Write samples at offsets \(s_jh\) from the target \(x\) as \(f(x+s_jh)\), where the offsets \(s_j\) and coefficients \(c_j\) are fixed. Consider the combination

Assume for now that \(f\) has enough derivatives near \(x\) for the Taylor expansions and remainders being used. Expanding a sample through degree \(m\) gives

After substituting these expansions into \(D_h f(x)\), the coefficient multiplying \(f^{(k)}(x)\) is \(h^{k-1}\sum_j c_js_j^k/k!\). To make the combination estimate \(f'(x)\), we require the constant term to vanish and the coefficient of \(f'(x)\) to equal one:

Further conditions can cancel higher derivatives. For example, requiring \(\sum_j c_js_j^2=0\) cancels the \(f''(x)\) term. These equations are the Taylor-matching conditions. Once the coefficients are chosen, the remainder terms must still be controlled; their size depends on the smoothness of \(f\).

Deriving the Forward Formula

Use the two samples \(f(x)\) and \(f(x+h)\). Write a candidate formula as \(\bigl(af(x)+bf(x+h)\bigr)/h\). The offsets are \(0\) and \(1\). The constant-term condition is \(a+b=0\), and the first-derivative condition is \(b=1\). Therefore \(a=-1\), giving the forward difference.

To describe its error term explicitly, suppose \(f\) has a bounded fourth derivative in a neighborhood of \(x\). Taylor's theorem then gives

Subtract \(f(x)\) and divide by \(h\):

Thus the forward formula retains a term involving \(f''(x)\), which is generally the leading difference between the estimate and the derivative.

Deriving the Backward Formula

Now use \(f(x-h)\) and \(f(x)\), writing the candidate as \(\bigl(af(x-h)+bf(x)\bigr)/h\). The offsets are \(-1\) and \(0\). Cancellation of the constant term gives \(a+b=0\); matching the first derivative gives \(-a=1\). Thus \(a=-1\), \(b=1\), and the backward difference is

Under the same assumption that \(f\) has a bounded fourth derivative near \(x\), expand the backward sample:

Subtract this expression from \(f(x)\), then divide by \(h\):

The sign of the term involving \(f''(x)\) is opposite to the one in the forward formula. Both formulas use values on only one side of \(x\), which can be useful when samples are available only on that side.

Deriving the Centered Formula

For a centered formula, use the samples \(f(x-h)\), \(f(x)\), and \(f(x+h)\). Let the numerator be \(af(x-h)+bf(x)+cf(x+h)\). Matching the constant term, the first derivative, and cancellation of the second derivative gives

The last equation gives \(c=-a\). Substitution into \(-a+c=1\) gives \(-2a=1\), so \(a=-1/2\) and \(c=1/2\). Then \(b=0\). The resulting centered formula is

The cancellation of the second-derivative term is why the centered formula has a different expansion from the one-sided formulas. To justify its next remainder precisely, assume \(f\) has a bounded fifth derivative near \(x\). Expand each sample through degree four; each Taylor remainder is then \(O(h^5)\):

Subtract the second expansion from the first. The constant, second-derivative, and fourth-derivative terms cancel. The two \(O(h^5)\) remainders have a difference that is still \(O(h^5)\), so

Dividing by \(2h\) gives

A Remainder Pitfall

Cancellation of displayed Taylor terms does not automatically mean that the remainder terms cancel to a higher order. Each remainder must be justified at the order claimed. For instance, expansions only through the third derivative with separate \(O(h^4)\) remainders justify at most an \(O(h^4)\) remainder after subtraction; they do not by themselves justify \(O(h^5)\).

For a concrete example, let \(f(t)=\max(t,0)^4\) and take \(x=0\). Its value and first three derivatives at zero are all zero. For \(h>0\), \(f(h)=h^4\) and \(f(-h)=0\). Each sample is consistent with a Taylor expansion through degree three with an \(O(h^4)\) remainder, but their difference is \(h^4\), which is not \(O(h^5)\). The centered derivative estimate is \(h^4/(2h)=h^3/2\), not \(O(h^4)\). To obtain the stronger centered remainder above, the proof instead assumes a bounded fifth derivative and expands each sample through degree four with an \(O(h^5)\) remainder before subtracting.

How the Three Formulas Differ

The table summarizes the displayed Taylor expansions under the smoothness assumptions stated above. It emphasizes the role of coefficient matching: the centered formula cancels the second-derivative term, while the one-sided formulas do not. The choice of formula also depends on which sample points are available. Questions about balancing approximation error against rounding effects are a separate issue.