Finite-Sample Covariance Diagnostics

Two Sample Autocovariances, Two Different Guarantees

For a finite record, the sample autocovariance at lag \(h\) is often computed with either a common divisor \(n\) or a lag-dependent divisor \(n-h\). The second convention compensates for the smaller number of observed pairs at larger lags, and it is often called the unbiased sample autocovariance. But the two conventions have different structural properties: the common-divisor estimates always form positive-semidefinite Toeplitz matrices, whereas the lag-corrected estimates need not. This matters when an estimated sequence is used as a candidate covariance sequence or as input to a procedure that requires a valid covariance matrix.

Let \((X_t)_{t\ge 0}\) be a real-valued weakly stationary process on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with finite second moments, mean \(\mu=\mathbb{E}X_t\), and autocovariance \(\gamma(h)=\operatorname{Cov}(X_{t+h},X_t)\). Observe \(X_1,\ldots,X_n\), where \(n\ge2\), and let \(\bar X_n=n^{-1}\sum_{t=1}^nX_t\). For \(h=0,\ldots,n-1\), define the centered sample autocovariances below; for negative \(h\), set each estimate equal to its value at \(|h|\).

The subscripts \(b\) and \(u\) denote the common-divisor, or biased, estimate and the lag-corrected estimate. At lag zero the formulas agree. At a positive lag, the second formula multiplies the first by \(n/(n-h)\). To form a Toeplitz matrix of order \(r\), use the estimated lags \(0,\ldots,r-1\): its \((j,k)\) entry is \(\widehat{\gamma}(j-k)\), for \(0\le j,k\le r-1\). A real symmetric matrix is positive semidefinite if \(a^\mathsf{T}Ma\ge0\) for every real vector \(a\). For covariance diagnostics, that condition is essential: every finite covariance matrix must be positive semidefinite.

Why the Common-Divisor Estimate Is Always Positive Semidefinite

Proof. Write \(y_t=x_t-\bar x_n\) for \(1\le t\le n\), and set \(y_t=0\) for every other integer \(t\). The zero extension gives, for every integer \(h\), \(\widehat{\gamma}_b(h)=n^{-1}\sum_{t\in\mathbb{Z}}y_{t+h}y_t\). This agrees with the sample formula for \(|h|<n\); for \(|h|\ge n\), there are no overlapping nonzero terms, so the sum is zero. Take any order \(r\ge1\) and any real vector \(a=(a_0,\ldots,a_{r-1})^\mathsf{T}\). Expanding its quadratic form and changing the summation index in each finite sum gives

In the second equality, for each pair \((j,k)\) the index change is \(s=t-k\); then \(y_{t+j-k}y_t=y_{s+j}y_{s+k}\). The sums are finite because \(y\) is zero outside \(1,\ldots,n\). The final expression is a sum of squares divided by \(n>0\), so it is nonnegative for every \(a\). This is exactly positive semidefiniteness. The argument also covers zero data and the zero vector \(a\), for which the quadratic form is zero. \(\square\)

This is a finite-record guarantee, not a claim that the estimated values equal the process autocovariances. Nor does it assert that the matrix must be positive definite: zero eigenvalues are allowed. The guarantee comes from representing the quadratic form as a sum of squares, so it applies at every matrix order, including orders larger than the observed record.

What “Unbiased” Does—and Does Not—Mean

The name “unbiased” has a precise justification if the true mean is known and used for centering. Define the known-mean estimate \(\widetilde{\gamma}_u(h)=(n-h)^{-1}\sum_{t=1}^{n-h}(X_{t+h}-\mu)(X_t-\mu)\), for \(0\le h<n\). Weak stationarity makes the expectation of each product equal to \(\gamma(h)\), so averaging the \(n-h\) products gives the result below.

Proof. By linearity of expectation and stationarity,

There are exactly \(n-h\) summands, and \(n-h>0\) over the stated range of lags. This proves the proposition. \(\square\)

When \(\mu\) is replaced by the observed sample mean, the proposition does not establish unbiasedness for \(\widehat{\gamma}_u(h)\): the centering value is then random and depends on all the observations. For instance, even at lag zero, sample-mean centering can introduce bias. If \(X_1,\ldots,X_n\) are independent with common mean \(\mu\) and variance \(\sigma^2\), then

Indeed, \(\sum_t(X_t-\bar X_n)^2=\sum_t(X_t-\mu)^2-n(\bar X_n-\mu)^2\). The expectation of the first sum is \(n\sigma^2\), and \(\operatorname{Var}(\bar X_n)=\sigma^2/n\), so the expectation of the subtracted term is \(\sigma^2\). Thus “unbiased” is best understood as the denominator correction for the known-mean estimator; with a sample mean, it is a conventional name, not a blanket unbiasedness guarantee.

Worked Toeplitz Diagnostics

Using the Diagnostic Carefully

A common mistake is to argue that the lag-corrected matrix must be positive semidefinite because each lag estimate uses all available pairs at that lag, or because the name “unbiased” sounds preferable. A concrete incorrect calculation is to take the first record above and use \(-4/3\) for the lag-one corrected estimate: that divides its numerator by \(n=3\), not by \(n-h=2\). The actual corrected lag-one value is \(-2\); with the lag-two value \(1\), the corrected order-three matrix has quadratic form \(-2\) at \(a=(1,2,1)^\mathsf{T}\). The correction changes the relative lag weights, so the sum-of-squares representation in the theorem no longer applies.

For a finite-sample covariance diagnostic, first specify which convention was used, then construct the Toeplitz matrix from the same estimated lags and test its quadratic forms or eigenvalues. A negative eigenvalue or one explicitly negative quadratic form rules out positive semidefiniteness. A successful test at one selected matrix order does not establish positive semidefiniteness at every order. Conversely, failure of a lag-corrected estimate is not evidence that the underlying process has an invalid covariance function: it is a property of this finite-sample estimate. The common-divisor convention preserves the covariance-matrix constraint automatically, while the lag-corrected convention targets a different finite-sample expectation when the mean is known.