High-Order Taylor Methods for Initial-Value Problems

From a Differential Equation to a One-Step Update

A scalar initial-value problem specifies a differential equation and a starting value, usually written as \(y'=f(t,y)\) and \(y(t_0)=y_0\). The solution is a function of time, but a numerical method advances by discrete steps: from an approximate value \(y_n\) at time \(t_n\), it computes an approximate value \(y_{n+1}\) at \(t_{n+1}=t_n+h\).

Euler’s method uses only the slope at the beginning of the step. A Taylor method uses more information: the curvature and higher derivatives of the solution at that point. Those derivatives are not usually supplied directly. They are obtained by differentiating the differential equation and applying the chain rule.

For a third-order method, the update uses the first three derivatives of the solution. For a fourth-order method, it uses the first four. Higher order means more derivative information and more computation per step; it does not mean that every step is automatically accurate regardless of the equation or step size.

Finding Solution Derivatives from \(y'=f(t,y)\)

The equation directly gives \(y'=f(t,y)\). To find \(y''\), differentiate both sides with respect to \(t\). Since \(f\) depends on \(t\) directly and on \(y(t)\), the chain rule gives

Here \(f_t\) and \(f_y\) denote partial derivatives, evaluated at the current \((t,y)\). This is a total derivative of \(f(t,y(t))\), not just its partial derivative with respect to \(t\).

Differentiate once more to obtain the third derivative:

All terms on the right are evaluated at the same point \((t,y)\). This formula follows by differentiating \(y''=f_t+f_yf\), applying the chain rule to each factor, and using \(y'=f\) and \(y''=f_t+f_yf\). It requires the relevant partial derivatives of \(f\) to exist.

For organizing higher derivatives, it is useful to define an operator \(D\) acting on a differentiable function \(q(t,y)\) by

Along a solution, the chain rule says \(\frac{d}{dt}q(t,y(t))=Dq\). Thus \(y'=f\), \(y''=Df\), \(y'''=D(Df)\), and \(y''''=D(D(Df))\). For a fourth-order update, one can differentiate the already-computed expressions as needed; the point is always to take derivatives along the solution, rather than to treat \(y\) as constant.

Why the Taylor Update Works

Fix a time \(t_n\), and consider the exact solution on the step. Set \(g(s)=y(t_n+s)\). Then \(g(0)=y(t_n)\), \(g'(0)=y'(t_n)\), and, more generally, \(g^{(k)}(0)=y^{(k)}(t_n)\). Taylor’s theorem applied to \(g\) gives the update formula and explains the role of each derivative.

The exact Taylor expansion through degree \(p\) is

If \(\lvert y^{(p+1)}(t)\rvert\le M\) throughout the step, the remainder satisfies \(\lvert R_{p+1}\rvert\le M\lvert h\rvert^{p+1}/(p+1)!\). This is the local truncation error when the step starts from the exact solution value. It describes one step; it does not by itself state the total error after many steps.

The Third- and Fourth-Order Updates

Using the derivative expressions from the differential equation, the third-order Taylor method is

The fourth-order method adds one more term:

In these formulas, \(y''\), \(y'''\), and \(y''''\) mean derivatives of the solution, expressed using \(f\) and its partial derivatives. They are not independent inputs to the problem. The formulas require \(f\) to be differentiable enough for the indicated solution derivatives to exist. To assert a remainder bound of order \(h^{p+1}\), one also needs the corresponding \((p+1)\)st solution derivative to exist and be bounded on the step.

Worked Examples

A Common Pitfall: Treating \(y\) as Constant

A frequent error is to compute \(y''\) by taking only the partial derivative \(f_t\). That omits the change in \(f\) caused by the changing value of \(y(t)\). For \(y'=y^2\), the equation has \(f_t=0\), but concluding that \(y''=0\) is incorrect.

At \(y(0)=1\), the correct value is \(y''(0)=2\), not \(0\). A second-order Taylor step with \(h=0.1\) would therefore be \(1+0.1+\frac{(0.1)^2}{2}(2)=1.11\). The incorrect calculation would give \(1+0.1=1.1\), because it drops the curvature term. The chain rule is essential even when \(f\) has no explicit dependence on \(t\).

Using the Method Carefully

High-order Taylor methods are useful when the differential equation is smooth and its derivatives can be calculated reliably. The method makes the connection between the local shape of the solution and the numerical update explicit: the slope sets the linear term, curvature contributes the quadratic term, and higher derivatives refine the step further.

The main practical cost is finding those higher derivatives. For a complicated \(f\), differentiating repeatedly may produce lengthy expressions, and errors in those expressions directly affect the update. The method also depends on adequate smoothness: if the required derivatives do not exist or are not controlled on the step, the Taylor remainder estimate cannot be used in the stated way.

Before carrying out a step, identify the desired order, compute each solution derivative along the differential equation, evaluate all quantities at the starting pair \((t_n,y_n)\), and then substitute the step size with its factorial coefficient. Keeping these stages separate helps prevent confusion between partial derivatives, total derivatives, and derivatives evaluated at different points.