Convexity Tests from Matrix Hessians

Curvature Along Lines Determines Convexity

For an objective depending on a vector $x$, the Hessian records curvature in every direction. The key link is that convexity can be checked by restricting the objective to each line segment in its domain. Along a line, the multivariate function becomes a one-variable function, and its second derivative is a quadratic form involving the Hessian. This gives a precise test: nonnegative curvature in every direction at every point is equivalent to convexity, under the regularity and domain conditions stated below.

This tutorial uses the Hessian of a scalar-valued function, as developed in the earlier material on second differentials and matrix Hessians. The focus here is how the Hessian’s sign determines the geometry of the objective, especially for quadratic objectives.

Definition: A set $D\subseteq\mathbb{R}^p$ is convex if, for every $x,y\in D$ and every $t\in[0,1]$, the point $(1-t)x+ty$ belongs to $D$. A function $f:D\to\mathbb{R}$ is convex if $$ f((1-t)x+ty)\leq (1-t)f(x)+t f(y) $$ for all $x,y\in D$ and $t\in[0,1]$. It is strictly convex if the inequality is strict whenever $x\ne y$ and $0<t<1$.

For a symmetric matrix $M$, positive semidefinite means $v^\mathsf{T}Mv\geq 0$ for every vector $v$; positive definite means $v^\mathsf{T}Mv>0$ for every nonzero vector $v$. Negative values in some direction indicate negative curvature in that direction. A $C^2$ function has a symmetric Hessian, denoted $\nabla^2 f(x)$, so these definitions apply directly.

Theorem: Hessian criterion for convexity. Let $D\subseteq\mathbb{R}^p$ be open and convex, and let $f:D\to\mathbb{R}$ be twice continuously differentiable. Then $f$ is convex on $D$ if and only if $\nabla^2 f(x)$ is positive semidefinite for every $x\in D$. If the Hessian is positive definite at every $x\in D$, then $f$ is strictly convex on $D$.

Why the Hessian Criterion Holds

Fix two points $x,y\in D$, and write $d=y-x$. Since $D$ is convex, the segment $x+td$, for $0\leq t\leq 1$, lies in $D$. Define the one-variable restriction $g(t)=f(x+td)$. By the chain rule, $$ g''(t)=d^\mathsf{T}\nabla^2 f(x+td)d. $$ Thus, a positive-semidefinite Hessian makes the restriction have nonnegative second derivative. Conversely, convexity of every such restriction forces its second derivative to be nonnegative. The following proof makes both implications precise.

Assume the Hessian is positive semidefinite.
For any segment in $D$, set $g(t)=f(x+t(y-x))$. Then $g''(t)=(y-x)^\mathsf{T}\nabla^2 f(x+t(y-x))(y-x)\geq 0$. By the mean value theorem, $g'$ is nondecreasing on the segment.

Deduce the convexity inequality.
For $0<t<1$, the mean value theorem gives a point $u\in(0,t)$ and a point $v\in(t,1)$ such that $(g(t)-g(0))/t=g'(u)$ and $(g(1)-g(t))/(1-t)=g'(v)$. Since $u<v$ and $g'$ is nondecreasing, the first slope is at most the second.

Rearrange.
From $(g(t)-g(0))/t\leq(g(1)-g(t))/(1-t)$, multiplying by $t(1-t)>0$ and collecting terms gives $g(t)\leq(1-t)g(0)+tg(1)$. This is the convexity inequality for $f$.

For the reverse implication, suppose $f$ is convex. Fix $x\in D$ and any direction $v\in\mathbb{R}^p$. Because $D$ is open, $x+sv$ lies in $D$ for all sufficiently small $|s|$. The function $h(s)=f(x+sv)$ is convex on this interval. Convexity at the midpoint gives $h(s)+h(-s)-2h(0)\geq 0$. For $s\ne 0$, divide by $s^2$ and let $s\to 0$. Twice differentiability implies that this symmetric second difference converges to $h''(0)$. The chain rule gives $h''(0)=v^\mathsf{T}\nabla^2 f(x)v$, which is therefore nonnegative. Since $x$ and $v$ were arbitrary, the Hessian is positive semidefinite everywhere.

If the Hessian is positive definite everywhere, then for distinct $x,y$, $d=y-x\ne 0$, and the same line restriction satisfies $g''(t)>0$ throughout the segment. Its derivative is strictly increasing, so for $0<t<1$ the first secant slope is strictly smaller than the second. Rearranging gives $g(t)<(1-t)g(0)+tg(1)$, proving strict convexity. $\square$

Quadratic Objectives and Their Hessians

For a quadratic objective, the Hessian is constant, so one matrix test settles convexity throughout the domain. Let $H$ be a symmetric $p\times p$ matrix, $b\in\mathbb{R}^p$, and $c\in\mathbb{R}$. For $$ q(x)=\frac{1}{2}x^\mathsf{T}Hx+b^\mathsf{T}x+c, $$ the gradient is $Hx+b$, and the Hessian is $H$. The linear and constant terms affect the gradient and function values but not curvature.

More generally, if the quadratic term is written $x^\mathsf{T}Ax$ for a matrix $A$ that is not necessarily symmetric, then $$ x^\mathsf{T}Ax=x^\mathsf{T}\left(\frac{A+A^\mathsf{T}}{2}\right)x, \qquad \nabla^2(x^\mathsf{T}Ax)=A+A^\mathsf{T}. $$ The antisymmetric part contributes zero to the scalar quadratic form. Thus, it is the symmetric Hessian—not an arbitrary matrix representation of the quadratic term—that must be tested.

Quadratic consequence: On $\mathbb{R}^p$, $q(x)=\frac{1}{2}x^\mathsf{T}Hx+b^\mathsf{T}x+c$, with $H=H^\mathsf{T}$, is convex if and only if $H$ is positive semidefinite. It is strictly convex if $H$ is positive definite. If $H$ has a direction $v$ with $v^\mathsf{T}Hv<0$, then $q$ is not convex on any open convex domain.

The final claim follows by restricting $q$ to a sufficiently short line through an interior point in direction $v$. The restricted function has second derivative $v^\mathsf{T}Hv<0$, contradicting the necessary Hessian condition for convexity.

For a symmetric $2\times2$ matrix $M=\begin{pmatrix}a&b\\b&c\end{pmatrix}$, a convenient test is that $M$ is positive semidefinite exactly when $a\geq0$, $c\geq0$, and $ac-b^2\geq0$. It is positive definite exactly when $a>0$ and $ac-b^2>0$. For the semidefinite test, these conditions make the quadratic form nonnegative: if $a>0$, write it as $a(u+(b/a)v)^2+(c-b^2/a)v^2$; if $a=0$, the determinant condition forces $b=0$, leaving $cv^2\geq0$. For the definite test, the same completion of the square proves strict positivity for every nonzero $(u,v)$. Conversely, positivity of the form requires the stated diagonal and determinant conditions, as is also seen from this quadratic form or its eigenvalues.

Worked Example: A Strictly Convex Quadratic

Consider $q(x,y)=x^2+2xy+2y^2-3x+y$. Its Hessian and directional quadratic form are $$ H=\begin{pmatrix}2&2\\2&4\end{pmatrix}, \qquad \begin{pmatrix}u&v\end{pmatrix}H\begin{pmatrix}u\\v\end{pmatrix} =2u^2+4uv+4v^2 =2(u+v)^2+2v^2. $$

The last expression is positive for every $(u,v)\ne(0,0)$: if it were zero, both $u+v=0$ and $v=0$, so $u=v=0$. Hence $H$ is positive definite. Equivalently, its upper-left entry is $2>0$ and its determinant is $2(4)-2^2=4>0$. The quadratic consequence shows that $q$ is strictly convex on $\mathbb{R}^2$. The linear terms $-3x+y$ do not change this conclusion.

Worked Example: A Convex but Not Strictly Convex Quadratic

Let $q(x,y)=(x+2y)^2$. Expanding and differentiating gives $$ q(x,y)=x^2+4xy+4y^2, \qquad H=\begin{pmatrix}2&4\\4&8\end{pmatrix}. $$ For any direction $(u,v)$, its Hessian quadratic form is $$ \begin{pmatrix}u&v\end{pmatrix}H\begin{pmatrix}u\\v\end{pmatrix} =2u^2+8uv+8v^2=2(u+2v)^2\geq0. $$ Thus $H$ is positive semidefinite. It is not positive definite: in direction $(u,v)=(-2,1)$, the form is zero, and $H\begin{pmatrix}-2\\1\end{pmatrix}=\begin{pmatrix}0\\0\end{pmatrix}$. Therefore $q$ is convex but not strictly convex. Indeed, along any line parallel to $(-2,1)$, the quantity $x+2y$, and hence $q$, stays constant. Distinct points on such a line satisfy equality in the convexity inequality.

Worked Example: An Indefinite Hessian Rules Out Convexity

Consider $q(x,y)=x^2+4xy+y^2$. Its Hessian is $$ H=\begin{pmatrix}2&4\\4&2\end{pmatrix}. $$ In the direction $(1,-1)$, the quadratic form is $2-8+2=-4<0$. Equivalently, restrict the objective to the line $(x,y)=(t,-t)$: $$ q(t,-t)=t^2-4t^2+t^2=-2t^2, \qquad \frac{d^2}{dt^2}q(t,-t)=-4. $$ The restriction has negative second derivative, so it is not convex on any interval of positive length. Consequently, $q$ is not convex on $\mathbb{R}^2$, nor on any open convex domain containing a segment in this direction.

Common Mistakes and Practical Interpretation

The Hessian test is valuable because it converts a geometric question into a direction-by-direction matrix calculation. For a quadratic objective, curvature does not vary with $x$, so the conclusion applies everywhere on the domain. In statistical optimization, this helps determine whether an objective has a convex shape before attempting to minimize it. Convexity alone does not identify a minimizer or guarantee that one exists; it describes the shape of the function.

Using positive definiteness as the convexity requirement. Positive semidefiniteness is enough for convexity. A zero-curvature direction may prevent strict convexity without destroying convexity, as the second worked example shows.
Checking only diagonal entries. Nonnegative diagonal entries do not ensure a positive-semidefinite matrix. For example, $\begin{pmatrix}1&2\\2&1\end{pmatrix}$ has positive diagonal entries but its quadratic form at $(1,-1)$ is $1-4+1=-2$.
Ignoring the domain condition. The criterion is a statement on a convex domain: the line segments used in the proof must remain in the domain. On a nonconvex set, a pointwise Hessian condition does not by itself establish the defining inequality between every pair of points in that set.
Testing an unsymmetrized quadratic matrix. For $x^\mathsf{T}Ax$, use the Hessian $A+A^\mathsf{T}$, or equivalently the symmetric part of $A$, rather than treating $A$ itself as the Hessian.

Key takeaway: For a $C^2$ function on an open convex domain, convexity is equivalent to a positive-semidefinite Hessian everywhere. Positive definiteness guarantees strict convexity; for quadratic objectives, these tests reduce to checking one constant symmetric matrix.

Check Your Understanding

Answer each question by referring to the Hessian criterion and, where relevant, checking the quadratic form.

State the domain and differentiability conditions in the Hessian criterion for convexity.
Why does restricting a multivariate function to a line segment help prove the Hessian criterion?
For $q(x,y)=x^2+2y^2$, find its Hessian and determine whether it is convex and strictly convex.
For $q(x,y)=(x-y)^2$, identify a nonzero direction with zero curvature and explain what that implies about strict convexity.
A symmetric Hessian has positive diagonal entries but a negative determinant. What can you conclude about convexity on an open convex domain?

Convexity Tests from Matrix Hessians

What You'll Learn