Taylor Expansion Behind the Second Derivative Test

Why a Quadratic Approximation Appears

A stationary point is a point where the first-order change vanishes. Near such a point, a differentiable function has no linear term in its local approximation. If the function is twice continuously differentiable, the leading possible change is therefore quadratic, and the Hessian records that change.

This is the mathematical reason the second derivative is central to local analysis. The quadratic approximation describes the function’s leading behavior near the point, while the remainder measures what the approximation leaves out. The size of that remainder—and the assumptions that guarantee its size—matter whenever we draw conclusions from the quadratic term.

The Second-Order Taylor Formula

Let \(f\) be a real-valued function defined on an open subset of \(\mathbb{R}^2\), and let \(\mathbf{a}=(a_1,a_2)\) be a point in its domain. Write a nearby point as \(\mathbf{a}+\mathbf{h}\), where \(\mathbf{h}=(h_1,h_2)\). The gradient \(\nabla f(\mathbf{a})\) is the vector of first partial derivatives, and the Hessian \(\mathbf{H}_f(\mathbf{a})\) is the matrix of second partial derivatives at \(\mathbf{a}\).

For a stationary point, the linear term is zero. The formula then takes the particularly useful form below. Here \(\|\mathbf{h}\|\) denotes the Euclidean length of the displacement, and \(r(\mathbf{h})=o(\|\mathbf{h}\|^2)\) means that \(r(\mathbf{h})/\|\mathbf{h}\|^2\) tends to zero as \(\mathbf{h}\) tends to \(\mathbf{0}\), with \(\mathbf{h}\ne\mathbf{0}\).

In two variables, if the Hessian entries at \(\mathbf{a}\) are \(f_{xx}(\mathbf{a})\), \(f_{xy}(\mathbf{a})\), \(f_{yx}(\mathbf{a})\), and \(f_{yy}(\mathbf{a})\), the quadratic term is

When the second partial derivatives are continuous, the mixed partials agree, so this becomes \(\frac{1}{2}f_{xx}(\mathbf{a})h_1^2+f_{xy}(\mathbf{a})h_1h_2+\frac{1}{2}f_{yy}(\mathbf{a})h_2^2\). The factor of one half applies to the whole Hessian quadratic form; it does not mean that every displayed cross term has a half coefficient after the equal mixed partials are combined.

Why the Formula Holds

The multivariable formula can be obtained from one-variable Taylor expansion along the straight line from \(\mathbf{a}\) to \(\mathbf{a}+\mathbf{h}\). Define \(g(t)=f(\mathbf{a}+t\mathbf{h})\) for \(0\le t\le1\). For sufficiently small \(\mathbf{h}\), this segment remains in the neighborhood where \(f\) is twice continuously differentiable.

Since \(\int_0^1(1-t)\,dt=1/2\), this yields

where the remainder is exactly

To see why this remainder is small relative to \(\|\mathbf{h}\|^2\), let \(\|\mathbf{M}\|\) denote the operator norm of a matrix \(\mathbf{M}\). The inequality \(|\mathbf{u}^{\top}\mathbf{M}\mathbf{u}|\le \|\mathbf{M}\|\|\mathbf{u}\|^2\) implies

Continuity of the Hessian makes the supremum tend to zero as \(\mathbf{h}\) tends to \(\mathbf{0}\). Thus \(r(\mathbf{h})=o(\|\mathbf{h}\|^2)\). At a stationary point, \(g'(0)=\nabla f(\mathbf{a})^{\top}\mathbf{h}=0\), proving the stated formula.

What the Remainder Guarantees

The statement \(r(\mathbf{h})=o(\|\mathbf{h}\|^2)\) says that the error is eventually small compared with the size of the quadratic scale. It does not say that the error is zero, and it does not, on its own, give a cubic error bound. A function can be twice continuously differentiable while its second derivatives approach their values at \(\mathbf{a}\) more slowly than linearly in the distance.

If \(f\) has continuous third partial derivatives in a neighborhood of \(\mathbf{a}\), then a stronger estimate is available on a sufficiently small closed ball: there is a constant \(C\) such that \(|r(\mathbf{h})|\le C\|\mathbf{h}\|^3\) there. This stronger conclusion needs the extra smoothness; it should not be inferred from the twice-continuous-differentiability assumption alone.

The remainder also explains when the quadratic term can control nearby values. If the quadratic form is positive definite, it is bounded below by \(c\|\mathbf{h}\|^2\) for some \(c>0\). Because \(r(\mathbf{h})/\|\mathbf{h}\|^2\) tends to zero, for sufficiently small nonzero \(\mathbf{h}\) the remainder has magnitude less than \((c/2)\|\mathbf{h}\|^2\). The full change is then positive. A negative definite quadratic form gives the corresponding negative change. If the quadratic form has directions where it is zero, this uniform comparison is unavailable; the remainder can matter in those directions.

Worked Examples

A Common Mistake: Overstating the Error

A frequent error is to assume that twice continuous differentiability automatically makes the remainder cubic. For instance, the function \(f(x,y)=(x^2+y^2)^{3/2}\) is twice continuously differentiable at the origin, with value, gradient, and Hessian all zero there. Its Taylor formula at the origin has remainder \(r(x,y)=(x^2+y^2)^{3/2}\). If \(\rho=\sqrt{x^2+y^2}\), then \(|r|/\rho^2=\rho\to0\), so this is indeed \(o(\rho^2)\). But \(|r|/\rho^3=1/\rho\), which is not bounded near zero. Thus the remainder is not \(O(\rho^3)\). The correct conclusion under a continuous Hessian is the little-o estimate; a cubic bound requires additional smoothness.

At a stationary point, the Hessian quadratic form is the leading second-order prediction of the change in the function. The Taylor remainder says precisely how closely that prediction matches the actual change. Keeping both parts in view prevents a useful approximation from being mistaken for an exact identity.