Hessian Matrices and Quadratic Forms

Why Second Derivatives Need a Matrix

A function of two variables can bend differently in different directions. Its first partial derivatives describe first-order changes along the coordinate directions; its second partial derivatives record how those rates change. Collecting the second partial derivatives in a matrix makes it possible to examine the combined curvature along any direction.

The matrix is called the Hessian. From it, a quadratic expression gives the second derivative of the function along a line through a point. Studying that expression lets us distinguish positive, negative, and mixed second-order curvature without considering only the \(x\)- and \(y\)-directions separately.

The Hessian Matrix

Let \(f(x,y)\) have continuous second partial derivatives in a neighborhood of a point \((a,b)\). The Hessian matrix of \(f\) at that point is the matrix whose entries are the second partial derivatives, evaluated at \((a,b)\).

Here, for example, \(f_{xy}\) means differentiate first with respect to \(x\), then with respect to \(y\). Because the second partial derivatives are continuous near \((a,b)\), the equality of mixed partials gives \(f_{xy}(a,b)=f_{yx}(a,b)\). Under this condition, the Hessian is symmetric: its entries on opposite sides of the main diagonal are equal.

The Hessian depends on the point as well as the function. For a function whose second partial derivatives vary with \(x\) and \(y\), the matrix generally changes when the point changes. It is therefore important to evaluate every entry at the same specified point before interpreting the matrix.

The Quadratic Form of the Hessian

A direction in the plane can be represented by a vector \(\mathbf{h}=(u,v)^\top\). The quadratic form associated with the Hessian at \((a,b)\) is obtained by multiplying the Hessian on the left by the transpose of the direction vector and on the right by the direction vector itself.

If the Hessian is symmetric, expanding the matrix product gives

The factor of \(2\) on the mixed term comes from two entries of the matrix. In detail, the first row contributes \(f_{xx}u^2+f_{xy}uv\), while the second row contributes \(f_{yx}uv+f_{yy}v^2\). Since \(f_{xy}=f_{yx}\), the two mixed contributions add to \(2f_{xy}uv\).

Why the Quadratic Form Measures Directional Curvature

To see why this expression represents second-order change in a direction, follow the straight line that starts at \((a,b)\) with direction \((u,v)\). Define the single-variable function \(g(t)=f(a+tu,b+tv)\). The chain rule first gives \(g'(t)=u f_x(a+tu,b+tv)+v f_y(a+tu,b+tv)\). Differentiating once more and applying the chain rule to each partial derivative gives

At \(t=0\), the point on the line is \((a,b)\). Continuous second partial derivatives give \(f_{xy}(a,b)=f_{yx}(a,b)\), so

This proves that the Hessian quadratic form is the second derivative along the chosen line at the starting point. If \(\mathbf{h}\) is a unit vector, \(g''(0)\) is the second directional derivative in that unit direction. If \(\mathbf{h}\) is not a unit vector, the value also reflects the length of the direction vector: replacing \(\mathbf{h}\) by \(c\mathbf{h}\) multiplies the quadratic form by \(c^2\).

Positive and Negative Curvature

The sign of the quadratic form describes the sign of the second derivative in a direction. If it is positive for every nonzero direction, the second-order curvature is positive in every direction. If it is negative for every nonzero direction, the second-order curvature is negative in every direction. These statements concern the quadratic form at the specified point; they do not, by themselves, assert that the function has a local minimum or maximum there.

Mixed Curvature and the Role of Direction

A quadratic form has mixed curvature when it takes both positive and negative values on direction vectors. This means that the second derivative along some lines is positive and along other lines is negative. Looking only along the coordinate axes can miss this behavior: the mixed term \(2f_{xy}uv\) matters when both components of the direction are nonzero.

Common Pitfall: Losing the Mixed Term

A frequent error is to read the diagonal entries of the Hessian and write \(f_{xx}u^2+f_{yy}v^2\), omitting the mixed partials. For the Hessian \(\begin{pmatrix}0&2\\2&0\end{pmatrix}\), that incorrect calculation would give \(Q(u,v)=0\) for every direction. It would therefore miss the positive value in direction \((1,1)\) and the negative value in direction \((1,-1)\).

The correction is to multiply the full matrix, including both off-diagonal entries:

Another important distinction is between curvature and classification. A Hessian records second-order directional behavior. To use it to determine whether a critical point is a local maximum or minimum requires additional results and conditions; the quadratic form alone should not be treated as a complete classification rule.

Why the Hessian Is Useful

The Hessian organizes all the second partial derivatives in one object, while its quadratic form tests how those derivatives combine in any chosen direction. This is useful because a surface may bend differently along different lines through the same point. The diagonal entries describe curvature along the coordinate directions, but the mixed entries affect directions between them.

For a reliable calculation, keep the point, matrix, and direction explicit: compute the second partial derivatives, evaluate them at the point, form the symmetric matrix, and then multiply by the direction vector on both sides. The resulting number is the second derivative along the line with that direction. Its sign describes curvature there, without claiming more than the calculation establishes.