Banded Covariance Models and Bandwidth

Why impose a covariance bandwidth?

For a vector of measurements, an unrestricted covariance matrix assigns a separate covariance to every pair of coordinates. When the coordinates have a meaningful order—such as measurements taken at successive times, locations along a sequence, or adjacent anatomical sites—it may be plausible that distant measurements have negligible linear association. A banded covariance model encodes this idea by restricting which covariances may be nonzero.

The restriction is on covariance, not on the ordering itself: changing the order of the coordinates changes which pairs count as nearby. Nor does banding require repeated values along a diagonal. That additional, different restriction belongs to another covariance family.

“Half-bandwidth” specifies the convention: \(q\) counts how far the nonzero entries may extend from the main diagonal in either direction. For example, \(q=1\) permits the main diagonal and the two immediately adjacent diagonals. At the boundary of a finite matrix, fewer than \(2q+1\) entries may occur in a row. If \(q=p-1\), every matrix entry is permitted, so the model imposes no zero restrictions.

What bandwidth says about association

The band condition concerns pairs of coordinates whose indices are more than \(q\) places apart. For such a pair, \(\operatorname{Cov}(X_i,X_j)=0\); when both variances are positive, their correlation is also zero. This is a statement about linear association. In a general distribution, zero covariance does not imply independence, and banding alone does not assert that the variables outside a local neighborhood are independent.

There is a useful special case. If \(X\) is jointly multivariate normal, then two subvectors with zero cross-covariance are independent. Thus, for a banded Gaussian covariance, two groups of coordinates are independent whenever every index in one group is more than \(q\) places from every index in the other. To see this, their joint covariance is block diagonal, and the multivariate normal density with block-diagonal covariance factors into the product of the two marginal densities. This implication relies on joint normality; it is not a general property of banded covariance matrices.

A covariance bandwidth should also not be read as a statement about conditional dependence. It specifies marginal covariances. In particular, covariance zeros and zeros in a precision matrix have different interpretations and are different model restrictions.

How many parameters does the band restriction save?

A symmetric \(p\times p\) covariance matrix has \(p\) diagonal entries. For each distance \(k=1,\ldots,q\), there are \(p-k\) distinct off-diagonal entries at that distance; symmetry determines their counterparts on the other side of the diagonal. Therefore, when \(0\leq q<p\), the number of free covariance entries before positive-definiteness restrictions is

This count follows by adding the diagonal entries and the distinct permitted covariances above the diagonal. Positive definiteness restricts the allowable numerical values, but it does not reduce this count of free coordinates in the model. For \(q=p-1\), the formula gives \(p(p+1)/2\), the unrestricted symmetric-matrix count. At \(q=0\), it gives \(p\): only the variances are free.

Sample-moment banding

Suppose \(X_1,\ldots,X_n\) are independent observations from a \(p\)-variate population with unknown mean. Write \(\bar X=n^{-1}\sum_{r=1}^n X_r\) and define the centered sample covariance with divisor \(n\) by

The divisor \(n\) is convenient for likelihood calculations. The usual unbiased sample covariance uses \(n-1\) instead; choosing that divisor rescales all entries and does not change whether a banded matrix is positive definite. A direct sample-moment estimate at bandwidth \(q\) keeps the entries of \(S\) inside the band and sets the others to zero. This is a simple entrywise construction, but positive definiteness is not guaranteed.

Gaussian likelihood uses the whole covariance structure

For a likelihood comparison, assume the observations are independent and identically distributed as \(N(\mu,\Sigma)\), where \(\mu\) is unrestricted and \(\Sigma\) is positive definite with half-bandwidth \(q\). Maximizing over \(\mu\) gives \(\hat\mu=\bar X\). Up to an additive constant and a positive factor, maximizing the remaining likelihood is equivalent to minimizing

over positive-definite banded \(\Sigma\). The trace term involves the inverse of the full covariance matrix. As a result, the likelihood estimates of permitted covariances are generally coupled to each other and to the sample covariances outside the band. Entrywise truncation is not, in general, the likelihood solution.

The score equations make this distinction explicit. For a symmetric perturbation \(H\) in an allowed covariance direction, the differential of \(Q\) is \[ dQ=\operatorname{tr}\left\{\left(\Sigma^{-1}-\Sigma^{-1}S\Sigma^{-1}\right)H\right\}. \] At an interior likelihood optimum, this differential is zero for every symmetric \(H\) supported on the allowed band. Thus the score matrix must have zero entries in the free covariance directions. This is not the same equation as setting the forbidden entries of \(S\) to zero.

Proof. Let \(c>0\) be the smallest eigenvalue of \(S\). Then \(S-cI\) is positive semidefinite. Since \(\Sigma^{-1}\) is positive definite, \[ \operatorname{tr}(S\Sigma^{-1}) \geq c\,\operatorname{tr}(\Sigma^{-1}). \] If \(\lambda_1,\ldots,\lambda_p\) are the eigenvalues of \(\Sigma\), it follows that \[ Q(\Sigma)\geq \sum_{i=1}^{p}\left(\log\lambda_i+\frac{c}{\lambda_i}\right). \] For each \(i\), the summand tends to \(+\infty\) as \(\lambda_i\) tends to zero or to infinity, and it is bounded below on \((0,\infty)\). Consequently, any sequence of matrices on which \(Q\) is bounded above must have all eigenvalues bounded away from zero and infinity. A minimizing sequence therefore has a subsequence converging to a positive-definite matrix. The fixed band constraints are closed under limits, and \(Q\) is continuous on the positive-definite matrices. The limit is in the banded model and attains the infimum. \(\square\)

Choosing and interpreting a bandwidth

A smaller bandwidth imposes more zero restrictions and uses fewer covariance parameters. This can stabilize estimation when the dimension is large relative to the sample size, but it also rules out associations at longer separations. A larger bandwidth is more flexible but requires estimating more entries. The choice should reflect the order and scientific meaning of the coordinates, as well as the available information in the data.

The bandwidth is a structural assumption, not a claim that all relationships within the band are strong or equal. It also does not automatically describe dependence in every aspect of a distribution: it specifies the covariance matrix only. Interpret the zero restrictions at the level of uncorrelatedness unless additional distributional assumptions, such as joint normality, justify an independence interpretation.

Common mistake

• Wrong reasoning: “The sample covariance is positive definite, so setting its distant entries to zero must leave a valid covariance matrix.”
• Correction: Positive definiteness depends jointly on all entries. The three-variable calculation above gives a positive-definite \(S\) whose bandwidth-one truncation has determinant \(-0.62\). Check positive definiteness after banding, or estimate within the positive-definite banded model, as likelihood does.
• Another overinterpretation: “A zero covariance means the variables are independent.” This is not generally true. It is justified for jointly normal subvectors when their cross-covariance block is zero.