Three-Variable Hessians and Curvature

Why a Hessian Describes Local Bending

A stationary point of a function of three variables is a point where all three first partial derivatives vanish. At such a point, the first-order change in the function disappears. The next contribution to change is described by the function’s second partial derivatives, collected in a matrix called the Hessian.

To understand what that matrix tells us, imagine moving a small displacement away from the stationary point. The Hessian gives a quadratic expression in the three components of that displacement. This expression measures the leading second-order change in the function. Its value depends on the direction of motion, so it describes how the function bends in different directions.

Central idea: At a stationary point, the quadratic form of the Hessian gives the second-order part of the function’s local change. It does not, by itself, describe the whole function away from that point.

The Three-Variable Hessian

Let $f(x,y,z)$ have continuous second partial derivatives in a neighborhood of a point $\mathbf{p}=(a,b,c)$. Its Hessian matrix at $\mathbf{p}$ is the matrix of second partial derivatives evaluated at that point:

\mathbf{H}_f(\mathbf{p})= \begin{pmatrix} f_{xx}(\mathbf{p}) & f_{xy}(\mathbf{p}) & f_{xz}(\mathbf{p})\\ f_{yx}(\mathbf{p}) & f_{yy}(\mathbf{p}) & f_{yz}(\mathbf{p})\\ f_{zx}(\mathbf{p}) & f_{zy}(\mathbf{p}) & f_{zz}(\mathbf{p}) \end{pmatrix}.

Here, for example, $f_{xy}$ means that we differentiate first with respect to $x$, then with respect to $y$. Since the second partial derivatives are continuous near $\mathbf{p}$, the mixed partial derivatives agree: $f_{xy}=f_{yx}$, $f_{xz}=f_{zx}$, and $f_{yz}=f_{zy}$. Thus the Hessian is symmetric.

A point $\mathbf{p}$ is stationary when the gradient is zero there:

\nabla f(\mathbf{p}) = \begin{pmatrix} f_x(\mathbf{p})\\ f_y(\mathbf{p})\\ f_z(\mathbf{p}) \end{pmatrix} = \begin{pmatrix} 0\\0\\0 \end{pmatrix}.

The Hessian can be evaluated at any point where the second partial derivatives exist. In this tutorial, the important use is to evaluate it at a stationary point, where it describes the leading local change after the first-order contribution vanishes.

The Quadratic Form of a Hessian

Write a displacement from $\mathbf{p}$ as $\mathbf{h}=(h_1,h_2,h_3)^\top$. If the Hessian at $\mathbf{p}$ is $\mathbf{H}$, its associated quadratic form is $\mathbf{h}^{\top}\mathbf{H}\mathbf{h}$. If

\mathbf{H}= \begin{pmatrix} A&B&C\\ B&D&E\\ C&E&F \end{pmatrix},

then multiplying out gives

\mathbf{h}^{\top}\mathbf{H}\mathbf{h} = Ah_1^2+Dh_2^2+Fh_3^2 +2Bh_1h_2+2Ch_1h_3+2Eh_2h_3.

The factors of $2$ on the mixed terms come from symmetry: for example, the matrix product contains both $Bh_1h_2$ and $Bh_2h_1$. The quadratic form’s contribution to the function’s second-order change includes an additional factor of $1/2$.

Formula: If $f$ has continuous second partial derivatives near a stationary point $\mathbf{p}$, then for small $\mathbf{h}$, $$ f(\mathbf{p}+\mathbf{h}) = f(\mathbf{p})+\frac{1}{2}\mathbf{h}^{\top}\mathbf{H}_f(\mathbf{p})\mathbf{h} +o(\|\mathbf{h}\|^2). $$ The first-order term is absent because $\nabla f(\mathbf{p})=\mathbf{0}$.

The remainder notation $o(\|\mathbf{h}\|^2)$ means that the remainder divided by $\|\mathbf{h}\|^2$ tends to zero as $\mathbf{h}$ tends to $\mathbf{0}$. So the quadratic expression gives the leading second-order change for sufficiently small displacements. It does not assert that the function is exactly quadratic.

Why the Quadratic Form Gives the Second-Order Change

The connection can be derived by examining the function along a line through the stationary point. Fix a direction $\mathbf{u}=(u_1,u_2,u_3)^\top$, and define the one-variable function $g(t)=f(\mathbf{p}+t\mathbf{u})$. By the chain rule, $g'(0)=\nabla f(\mathbf{p})^\top\mathbf{u}=0$. Differentiating again gives

g''(0)=\mathbf{u}^{\top}\mathbf{H}_f(\mathbf{p})\mathbf{u}.

Because $f$ has continuous second partial derivatives, the one-variable Taylor expansion of $g$ at $0$ gives $g(t)=g(0)+\frac{1}{2}t^2g''(0)+o(t^2)$. Substituting the expression for $g''(0)$ yields

f(\mathbf{p}+t\mathbf{u}) = f(\mathbf{p})+ \frac{1}{2}t^2\mathbf{u}^{\top}\mathbf{H}_f(\mathbf{p})\mathbf{u} +o(t^2).

This shows why the Hessian measures directional bending. For a unit direction $\mathbf{u}$, the quantity $\mathbf{u}^{\top}\mathbf{H}_f(\mathbf{p})\mathbf{u}$ is the second derivative of the function along that direction at the stationary point. The quadratic form combines the effects of the three coordinate directions and their mixed interactions.

Worked Examples

Worked Example: Constructing the Hessian at the Origin

Consider $f(x,y,z)=x^2+2xy+3y^2+z^2$. First calculate the first partial derivatives:

f_x=2x+2y,\qquad f_y=2x+6y,\qquad f_z=2z.

At $\mathbf{p}=(0,0,0)$, these are $0,0,0$, so the origin is stationary. The second partial derivatives are

f_{xx}=2,\quad f_{xy}=2,\quad f_{xz}=0,\quad f_{yy}=6,\quad f_{yz}=0,\quad f_{zz}=2.

Therefore, the Hessian at the origin is

\mathbf{H}_f(\mathbf{0})= \begin{pmatrix} 2&2&0\\ 2&6&0\\ 0&0&2 \end{pmatrix}.

For a displacement $\mathbf{h}=(a,b,c)^\top$, the quadratic form is

\mathbf{h}^{\top}\mathbf{H}_f(\mathbf{0})\mathbf{h} =2a^2+4ab+6b^2+2c^2.

Thus its second-order contribution is $a^2+2ab+3b^2+c^2$. Checking against the function itself, $f(a,b,c)-f(0,0,0)=a^2+2ab+3b^2+c^2$, exactly as the formula gives. The function is quadratic, so there is no higher-order remainder.

Worked Example: A Hessian at a Nonzero Stationary Point

Use shifted coordinates $u=x-1$, $v=y+1$, and $w=z-2$, and define

$$ f(x,y,z)=u^2+2uv+3v^2+2w^2+u^2w. $$

The point $\mathbf{p}=(1,-1,2)$ corresponds to $u=v=w=0$. Differentiating with respect to the shifted coordinates gives

f_u=2u+2v+2uw,\qquad f_v=2u+6v,\qquad f_w=4w+u^2.

All three derivatives vanish at $u=v=w=0$, so $\mathbf{p}$ is stationary. The second partial derivatives there are $f_{uu}=2$, $f_{uv}=2$, $f_{uw}=0$, $f_{vv}=6$, $f_{vw}=0$, and $f_{ww}=4$. Hence

\mathbf{H}_f(\mathbf{p})= \begin{pmatrix} 2&2&0\\ 2&6&0\\ 0&0&4 \end{pmatrix}.

Consider displacements $\mathbf{h}=t(1,-1,2)^\top$. The quadratic form in the direction $\mathbf{u}=(1,-1,2)^\top$ is

\mathbf{u}^{\top}\mathbf{H}_f(\mathbf{p})\mathbf{u} =2(1)^2+4(1)(-1)+6(-1)^2+4(2)^2 =2-4+6+16=20.

The second-order contribution for displacement $t\mathbf{u}$ is therefore $\frac12t^2(20)=10t^2$. Direct substitution gives $u=t$, $v=-t$, and $w=2t$, so

f(\mathbf{p}+t\mathbf{u})-f(\mathbf{p}) =t^2-2t^2+3t^2+8t^2+2t^3 =10t^2+2t^3.

The leading term is $10t^2$, as predicted; the additional $2t^3$ is a higher-order contribution. Differentiating this exact expression twice with respect to $t$ and evaluating at $t=0$ gives $20$, confirming the directional second derivative computed from the Hessian.

Worked Example: Evaluating a Direction with Mixed Terms

Let

$$ f(x,y,z)=x^2+2xy+3xz+y^2+2yz+2z^2. $$

Its gradient is $\nabla f=(2x+2y+3z,\;2x+2y+2z,\;3x+2y+4z)^\top$, which is zero at the origin. The Hessian there is

\mathbf{H}_f(\mathbf{0})= \begin{pmatrix} 2&2&3\\ 2&2&2\\ 3&2&4 \end{pmatrix}.

For $\mathbf{h}=(1,2,-1)^\top$, first multiply the matrix by the vector:

\mathbf{H}_f(\mathbf{0})\mathbf{h} = \begin{pmatrix} 2+4-3\\ 2+4-2\\ 3+4-4 \end{pmatrix} = \begin{pmatrix} 3\\4\\3 \end{pmatrix}.

Then take the dot product with $\mathbf{h}$:

\mathbf{h}^{\top}\mathbf{H}_f(\mathbf{0})\mathbf{h} =1(3)+2(4)+(-1)(3)=3+8-3=8.

The second-order contribution is $\frac12(8)=4$. Checking directly, $f(1,2,-1)=1+4-3+4-4+2=4$, while $f(0,0,0)=0$. The result agrees; as in the first example, the function is exactly quadratic.

A Common Mixed-Term Error

A frequent mistake is to include each mixed partial derivative only once when expanding the quadratic form. For example, for a symmetric Hessian with $H_{12}=H_{21}=B$, a mistaken expansion might write only $Bh_1h_2$. But the matrix product contains both entries:

$$ Bh_1h_2+Bh_2h_1=2Bh_1h_2. $$

The factor of $2$ belongs in $\mathbf{h}^{\top}\mathbf{H}\mathbf{h}$. The Taylor formula then multiplies the entire quadratic form by $\frac12$. In the third worked example, the $xy$ terms contribute $2(2)(1)(2)=8$ to the quadratic form. Omitting one of the two symmetric entries would give $4$ for that contribution and an incorrect total.

Keep the factors distinct: The mixed terms are doubled when the matrix product is expanded. The full quadratic form is then multiplied by $\frac12$ in the Taylor approximation.

What the Hessian Does—and Does Not—Tell You

The Hessian at a stationary point summarizes the function’s second-order behavior there. Its quadratic form gives the leading change along each small displacement, accounting for both pure-coordinate effects and interactions between coordinates. This is why it is useful to study the matrix rather than only the three pure second partial derivatives.

The approximation is local: it concerns displacements close to the stationary point. If higher-order terms are present, the Hessian does not capture them. Also, computing the Hessian and its quadratic form is not the same as deciding the point’s full local classification. That requires a separate test and attention to the test’s conditions.

When using the formula, verify the point is stationary before dropping the linear term, use second partial derivatives evaluated at that point, and include the factor $\frac12$ when converting the quadratic form into the second-order change.

Check Your Understanding

Use the definitions and calculations in this tutorial to answer the following.

What condition on the second partial derivatives ensures that the Hessian is symmetric near a point?
For a symmetric Hessian, why does a mixed term appear twice in $\mathbf{h}^{\top}\mathbf{H}\mathbf{h}$?
Why does the first-order term vanish in the Taylor expansion at a stationary point?
What does $\mathbf{u}^{\top}\mathbf{H}_f(\mathbf{p})\mathbf{u}$ measure when $\mathbf{u}$ is a direction vector?
In the second worked example, which term accounts for the difference between the exact change and the second-order contribution?