Positive Definite Matrices and Definiteness Tests

Why Positive Definiteness Has Several Tests

A symmetric matrix is positive definite when every nonzero direction has strictly positive quadratic form. In statistical work, this condition distinguishes a genuinely positive amount of curvature in every direction from a flat direction or one that curves downward. It also provides several equivalent ways to check a matrix: through its eigenvalues, its quadratic form, or determinants of selected submatrices.

The determinant tests are easy to confuse. Requiring every principal minor to be positive is equivalent to positive definiteness. Sylvester’s criterion is a more economical test: it requires only the leading principal minors to be positive. These tests apply here to real symmetric matrices.

The Eigenvalue Characterization

The spectral theorem says that a real symmetric matrix has an orthonormal eigenbasis. Thus \(A=Q\Lambda Q^T\), where \(Q^TQ=I\) and \(\Lambda\) is diagonal with the eigenvalues \(\lambda_1,\ldots,\lambda_n\). For any \(x\), putting \(y=Q^Tx\) gives \(x^TAx=\sum_{i=1}^n\lambda_i y_i^2\). Since \(Q^T\) preserves whether a vector is zero, this identity gives an exact test.

Proof. If every \(\lambda_i>0\), then for any nonzero \(x\), the vector \(y=Q^Tx\) is nonzero. At least one \(y_i\) is nonzero, so \(\sum_i\lambda_i y_i^2>0\). Hence \(x^TAx>0\). Conversely, if \(A\) is positive definite, take any unit eigenvector \(v_i\) with eigenvalue \(\lambda_i\). Then \(\lambda_i=v_i^TAv_i>0\). This holds for every eigenvalue. \(\square\)

Principal Minors: The Full Determinant Test

For a symmetric matrix, positive definiteness is equivalent to positivity of all its principal minors, not merely its determinant. To see why the condition includes every principal submatrix, note first that if \(A\) is positive definite, any principal submatrix is positive definite: extend a nonzero vector on its selected coordinates by zeros, and evaluate the quadratic form of \(A\). A positive definite symmetric submatrix has positive eigenvalues, so its determinant, the product of those eigenvalues, is positive.

Proof of the converse. Suppose all principal minors of \(A\) are positive. Let the eigenvalues of \(A\) be \(\lambda_1,\ldots,\lambda_n\); they are real by symmetry. Expand \(\det(tI+A)\) as a polynomial in \(t\). Its coefficient of \(t^{n-k}\) is the sum of the principal minors of order \(k\): in the determinant expansion, choosing \(t\) from \(n-k\) diagonal positions leaves the determinant of the corresponding order-\(k\) principal submatrix. The leading coefficient is \(1\), and all other coefficients are strictly positive by assumption. Therefore \(\det(tI+A)>0\) for every \(t\geq 0\).

But \(\det(tI+A)=\prod_{i=1}^n(t+\lambda_i)\). If any eigenvalue were zero, this product would be zero at \(t=0\). If any eigenvalue were negative, it would be zero at the positive value \(t=-\lambda_i\). Both possibilities contradict strict positivity for \(t\geq0\). Thus all eigenvalues are positive, and the eigenvalue characterization proves that \(A\) is positive definite. \(\square\)

Sylvester’s Criterion: Only the Leading Minors

Checking every principal minor can require many determinants. Sylvester’s criterion reduces the work to \(n\) determinants. Write \(\Delta_k\) for the leading principal minor of order \(k\), and set \(\Delta_0=1\).

Proof. If \(A\) is positive definite, each leading principal submatrix is positive definite by restricting the quadratic form to vectors whose remaining coordinates are zero. Its eigenvalues are positive, so its determinant \(\Delta_k\) is positive.

For the converse, use induction on the matrix dimension. The result is immediate in dimension one. Suppose all leading minors of an \(n\times n\) symmetric matrix are positive. Its leading \((n-1)\times(n-1)\) submatrix \(B\) has all its leading minors positive, so it is positive definite by induction. Write \(A=\begin{pmatrix}B&b\\b^T&c\end{pmatrix}\). Since \(B\) is positive definite, it is invertible. Set \(\alpha=c-b^TB^{-1}b\). Multiplying \(A\) on the left by the determinant-one matrix \(\begin{pmatrix}I&0\\-b^TB^{-1}&1\end{pmatrix}\) gives a block upper-triangular matrix with diagonal blocks \(B\) and \(\alpha\). Consequently, \[ \Delta_n=\det(A)=\det(B)\alpha=\Delta_{n-1}\alpha. \] The assumptions give \(\Delta_n>0\) and \(\Delta_{n-1}>0\), so \(\alpha>0\). For any \(x\in\mathbb{R}^{n-1}\) and \(t\in\mathbb{R}\), direct expansion yields \[ \begin{pmatrix}x\\t\end{pmatrix}^{T}A\begin{pmatrix}x\\t\end{pmatrix} =(x+B^{-1}bt)^TB(x+B^{-1}bt)+\alpha t^2. \] Both terms are nonnegative. If \(t\ne0\), the second is strictly positive. If \(t=0\) and \(x\ne0\), the first is strictly positive because \(B\) is positive definite. Thus the quadratic form is positive for every nonzero \((x,t)\), completing the induction. \(\square\)

Choosing and Interpreting a Test

The eigenvalue test is conceptually direct, and the quadratic form describes what the condition means geometrically. For exact calculations, Sylvester’s criterion is often efficient because it uses only leading principal minors. The all-principal-minors theorem is useful as a characterization, but can require checking many more determinants. In dimension \(n\), there are \(2^n-1\) nonempty principal minors, compared with only \(n\) leading minors.

A common mistake is to infer positive definiteness from a positive determinant, or from a positive determinant together with positive diagonal entries. The matrix \(C\) above disproves both shortcuts. The correction is to use a complete criterion: check all eigenvalues, all principal minors, or all leading principal minors. For a numerical matrix, approximate eigenvalues or determinants near zero also require care: rounding can obscure whether a value is truly positive.