Testing Candidate Covariance Sequences

Why Test a Candidate Sequence?

A proposed covariance sequence may look plausible term by term and still be impossible: the values must work together as covariances of every finite collection of observations. The practical test is to build the covariance matrix for a finite block and check that it is positive semidefinite. This tutorial develops that test through Toeplitz structure, principal minors, and finite quadratic forms. A failed finite test rules out the candidate; passing a few tests does not establish that every required matrix is positive semidefinite.

Let \((X_n)_{n\in\mathbb{Z}}\) be a real-valued weakly stationary process on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with state space \(S=\mathbb{R}\). Weak stationarity means that \(\mathbb{E}[X_n]\) is a finite constant independent of \(n\), and that \(\operatorname{Cov}(X_{n+h},X_n)\) depends only on the lag \(h\). Write \(\gamma(h)=\operatorname{Cov}(X_{n+h},X_n)\). For a real-valued process, \(\gamma(-h)=\gamma(h)\). A candidate sequence is a proposed list of real numbers \((\gamma(h))_{h\in\mathbb{Z}}\) with this symmetry.

The matrix is called Toeplitz because its entries are constant along each diagonal: the entry depends on \(i-j\), not on \(i\) and \(j\) separately. If the candidate really is a covariance sequence, then \(T_m\) is the covariance matrix of \((X_0,\ldots,X_m)^{\mathsf T}\). Thus it must be positive semidefinite. Indeed, the quadratic form is the variance of a linear combination of observations.

What the Small Matrices Require

The order-zero matrix is just \([\gamma(0)]\), so \(\gamma(0)\ge 0\) is necessary. At order one, the matrix is \(\begin{bmatrix}\gamma(0)&\gamma(1)\\\gamma(1)&\gamma(0)\end{bmatrix}\). Its determinant must be nonnegative, giving \(\gamma(0)^2-\gamma(1)^2\ge 0\). More generally, using the observations at times \(0\) and \(k\) gives the same matrix with \(\gamma(k)\) in the off-diagonal entries. Consequently, every candidate must satisfy

These inequalities are useful screening checks, but they only test two observations at a time. A sequence can satisfy all of them and still fail a larger matrix test. For larger matrices, one systematic method is to inspect principal minors. A principal submatrix is formed by choosing the same set of indices for rows and columns; its determinant is a principal minor. Positive semidefiniteness requires every principal minor to be nonnegative. For a symmetric matrix, this condition is also sufficient, including when some minors are zero.

Proof. Suppose first that \(A\) is positive semidefinite. Every principal submatrix \(A_I\), indexed by a set \(I\), is positive semidefinite: for any vector on \(I\), extend it by zeros outside \(I\) and use the quadratic-form inequality for \(A\). By the spectral theorem, the eigenvalues of the symmetric matrix \(A_I\) are nonnegative. Its determinant, the product of those eigenvalues, is therefore nonnegative. This proves necessity.

For sufficiency, suppose every principal minor of \(A\) is nonnegative. For \(\varepsilon>0\), consider \(A+\varepsilon I\). For any index set \(I\), expansion of the determinant by multilinearity gives \(\det(A_I+\varepsilon I)=\sum_{J\subseteq I}\varepsilon^{|I|-|J|}\det(A_J)\), where the determinant for the empty set is defined to be \(1\). Each term is nonnegative, and the term for \(J=\varnothing\) is \(\varepsilon^{|I|}>0\). Thus every leading principal minor of \(A+\varepsilon I\) is positive.

To see why this makes \(A+\varepsilon I\) positive definite, use induction on its size. The size-one case follows from its positive leading minor. At a larger size, write the matrix in block form \(\begin{bmatrix}B&b\\b^{\mathsf T}&c\end{bmatrix}\), where \(B\) is the leading block of one smaller size. By induction, \(B\) is positive definite and invertible. The determinant identity gives \(c-b^{\mathsf T}B^{-1}b=\det(A+\varepsilon I)/\det(B)>0\), with the determinants here taken for the full matrix and for \(B\). For any \(u\) and scalar \(v\), its quadratic form is \((u+B^{-1}bv)^{\mathsf T}B(u+B^{-1}bv)+(c-b^{\mathsf T}B^{-1}b)v^2\). This is positive for every nonzero \((u,v)\), so \(A+\varepsilon I\) is positive definite. Finally, for every \(x\), \(x^{\mathsf T}Ax=\lim_{\varepsilon\downarrow0}x^{\mathsf T}(A+\varepsilon I)x\ge0\). Therefore \(A\) is positive semidefinite. \(\square\)

For covariance testing, the theorem means that one can test a finite Toeplitz matrix by checking all of its principal minors, not just the leading ones. Strictly positive leading principal minors are a useful test for positive definiteness, but covariance matrices are allowed to be singular. When a minor is zero, the nonnegative-minor criterion still applies.

Worked Tests

A Reliable Testing Procedure

For a candidate sequence, work from small matrices upward, using exact arithmetic where possible. At each order, write the full Toeplitz matrix before calculating; this reduces errors in the lag indices. Check the simple constraints \(\gamma(0)\ge0\) and \(|\gamma(k)|\le\gamma(0)\), then compute principal minors or exhibit a vector whose quadratic form is negative. A single negative principal minor or a single negative quadratic form is conclusive: the candidate cannot be a covariance sequence.

Do not interpret a successful finite check too broadly. Passing the order-one test says only that every two-point matrix of the tested form passes. Passing through order \(m\) checks blocks of at most \(m+1\) consecutive observations; it does not establish positive semidefiniteness at every order. A proposed sequence is not certified by a few reassuring determinants. Conversely, a negative test at any finite order settles the question immediately.

Common mistake. A tempting but incorrect shortcut is to check only \(|\gamma(k)|\le\gamma(0)\) and declare the candidate valid. In the second example, the checked values are \(\gamma(0)=1\), \(|\gamma(1)|=\tfrac34\le1\), and \(|\gamma(2)|=0\le1\), yet the order-two determinant is \(-\tfrac18\) and the explicit quadratic form is \(-\tfrac14\). The correction is to test joint consistency using the full finite covariance matrix, not just separate pairs of values.