Valid Correlation Matrices

What Makes a Correlation Matrix Valid?

A list of pairwise correlations can look reasonable one pair at a time and still be impossible as a joint description of random variables. The key restriction is not merely that each correlation lies between \(-1\) and \(1\): the entire matrix must also satisfy a joint consistency condition. That condition is positive semidefiniteness.

Suppose \(X_1,\ldots,X_p\) have finite, strictly positive variances. Their correlation matrix records the correlation of every pair, including each variable's correlation with itself. Thus its diagonal entries are all \(1\). The earlier material on covariance matrices establishes the basic quadratic-form condition behind the second requirement: any linear combination of random variables has nonnegative variance.

The full characterization is especially useful: it tells us both how to reject an impossible candidate and when the matrix conditions alone guarantee that some joint distribution can realize it.

The Characterization

For necessity, write \(Z_i=(X_i-E[X_i])/\sqrt{\operatorname{Var}(X_i)}\). Each \(Z_i\) has mean \(0\) and variance \(1\), and \(R_{ij}=E[Z_iZ_j]\). The matrix is symmetric because \(E[Z_iZ_j]=E[Z_jZ_i]\), and its diagonal entries are \(E[Z_i^2]=1\). For any \(a\in\mathbb{R}^p\),

This proves that \(R\) is positive semidefinite.

For sufficiency, suppose \(R\) is symmetric, positive semidefinite, and has unit diagonal. By the spectral theorem, there is a real matrix \(U\) such that \(R=UU^{\mathsf T}\). Let \(u_i\) denote row \(i\) of \(U\). Then \(u_i^{\mathsf T}u_j=R_{ij}\), and \(u_i^{\mathsf T}u_i=R_{ii}=1\). Take independent random variables \(\varepsilon_1,\ldots,\varepsilon_p\) with \(P(\varepsilon_k=1)=P(\varepsilon_k=-1)=1/2\), and define \(X_i=\sum_{k=1}^p U_{ik}\varepsilon_k\). Since \(E[\varepsilon_k]=0\) and \(E[\varepsilon_k\varepsilon_l]\) is \(1\) if \(k=l\) and \(0\) otherwise,

In particular, \(\operatorname{Var}(X_i)=R_{ii}=1\). Thus the correlations of these variables are exactly the entries of \(R\), proving sufficiency. \(\square\)

Testing Positive Semidefiniteness With Principal Minors

A principal submatrix is formed by selecting the same set of indices for its rows and columns. Its determinant is a principal minor. For a symmetric matrix, checking every principal minor gives an exact test, including when the matrix is singular.

If \(A\) is positive semidefinite, each principal submatrix is also positive semidefinite: its quadratic form is the quadratic form of \(A\) applied to vectors that are zero outside the selected indices. Its eigenvalues are therefore nonnegative, so its determinant, the product of those eigenvalues, is nonnegative.

For the converse, suppose every principal minor of an \(n\times n\) symmetric matrix \(A\) is nonnegative. Expanding the determinant gives

where \(c_k\) is the sum of all principal minors of order \(k\). Hence every \(c_k\geq 0\), and \(\det(tI+A)>0\) for every \(t>0\), since its leading term is strictly positive and all terms are nonnegative. If \(A\) had a negative eigenvalue \(\lambda\), then \(t=-\lambda>0\) would satisfy \(\det(tI+A)=0\), a contradiction. Since \(A\) is symmetric, all its eigenvalues are real; they must therefore all be nonnegative. This is exactly positive semidefiniteness. \(\square\)

For a correlation matrix, the principal minors of order one are already \(1\). A candidate must still pass the tests for every larger principal submatrix. In practice, listing every minor becomes cumbersome for large matrices, but the criterion explains what validity means and provides direct checks for small matrices.

The Three-Variable Condition

For three variables, write the correlations as \(a=R_{12}\), \(b=R_{13}\), and \(c=R_{23}\). The matrix has the form

Its three principal minors of order two are \(1-a^2\), \(1-b^2\), and \(1-c^2\). Nonnegativity of these minors gives the familiar pairwise restrictions \(|a|,|b|,|c|\leq 1\). The determinant is

Thus, together with the pairwise restrictions, the determinant must be nonnegative. If \(a\) and \(b\) are fixed, the determinant condition can also be written as

This is the exact range of values of \(c\) compatible with \(a\) and \(b\), when \(|a|\leq1\) and \(|b|\leq1\). The range describes joint compatibility, not just the separate bounds on each correlation.

Common Mistake: Checking Only the Leading Minors

A tempting shortcut is to check only the minors in the upper-left corner, sometimes called the leading principal minors. Nonnegative leading principal minors are not, in general, enough to establish positive semidefiniteness. For example,

has leading principal minors \(1\), \(0\), and \(0\), all nonnegative. But the principal minor using rows and columns \(1\) and \(3\) is

So \(A\) is not positive semidefinite. The correction is to check all principal minors when using the semidefinite criterion. Another distinction matters at the boundary: a zero determinant does not automatically mean a matrix is invalid. The third worked example shows a valid correlation matrix with determinant zero.

Why the Conditions Matter

A correlation matrix summarizes a collection of pairwise relationships while representing one joint set of variables. Positive semidefiniteness enforces consistency across every linear combination, not just across individual pairs. This is why the matrix criterion is useful when assessing a proposed dependence structure: it rules out combinations that cannot arise from random variables.