Tapering Estimators for Bandable Covariance Matrices

Why Taper Rather Than Keep or Delete

When variables have a meaningful order, nearby variables may be strongly correlated while distant variables tend to be less so. Covariance banding uses that order by setting sufficiently distant entries to zero. But abruptly deleting entries can make a valid covariance estimate indefinite. Tapering offers a different choice: it multiplies entries by weights that decrease with distance, preserving the more local structure while reducing the influence of distant, noisy estimates.

The taper width controls how far the estimate reaches. A wider taper retains more covariance information, but also retains more estimation noise. A narrower taper typically reduces variance, at the cost of more bias. The construction below makes that tradeoff explicit and, with a positive semidefinite window, ensures the resulting estimator is itself positive semidefinite.

Definition and a Positive Semidefinite Window

The window equals one on the diagonal, decreases linearly with distance, and is zero when \(|i-j|\geq k+1\). Thus entries at separation \(d\) are multiplied by \((1-d/(k+1))\) for \(1\leq d\leq k\). This Bartlett window is positive semidefinite: it is a Gram matrix. To see this, put \(L=k+1\) and, for each index \(i\), form a vector \(u_i\) whose coordinates are \(1/\sqrt L\) on the integer interval from \(i\) to \(i+L-1\), and zero elsewhere. The overlap of two such intervals gives $$ \langle u_i,u_j\rangle=\frac{(L-|i-j|)_+}{L}=(T_k)_{ij}. $$ Every Gram matrix is positive semidefinite, so \(T_k\succeq0\).

For completeness, write \(A_{ij}=\langle a_i,a_j\rangle\) and \(B_{ij}=\langle b_i,b_j\rangle\), using Gram representations of the two positive semidefinite matrices. The tensor-product vectors \(a_i\otimes b_i\) have inner products $$ \langle a_i\otimes b_i,a_j\otimes b_j\rangle =\langle a_i,a_j\rangle\langle b_i,b_j\rangle =A_{ij}B_{ij}. $$ Therefore \(A\circ B\) is also a Gram matrix and is positive semidefinite. Since \(S\) is a sum of outer products, it is positive semidefinite; applying the result with \(A=T_k\) and \(B=S\) proves the claim. \(\square\)

Bias and Variance Under Frobenius Loss

For a matrix \(A\), write \(\|A\|_F^2=\sum_{i,j}A_{ij}^2\). The following exact risk identity separates the systematic change made by the window from the sampling variability of \(S\).

To prove it, decompose the estimation error entry by entry: $$ (\widehat\Sigma_k-\Sigma)_{ij} =((T_k)_{ij}-1)\Sigma_{ij} +(T_k)_{ij}(S_{ij}-\Sigma_{ij}). $$ The first term is deterministic. The second has expectation zero because \(E[S_{ij}]=\Sigma_{ij}\); hence the cross term has expectation zero. The squared deterministic term gives the first sum in the identity. For the random term, \(S_{ij}=n^{-1}\sum_{a=1}^n X_{ai}X_{aj}\). Independence across observations and the Gaussian fourth-moment identity give $$ \operatorname{Var}(S_{ij}) =\frac{1}{n}\operatorname{Var}(X_{1i}X_{1j}) =\frac{1}{n}\bigl(\Sigma_{ii}\Sigma_{jj}+\Sigma_{ij}^2\bigr). $$ Multiplying each variance by \((T_k)_{ij}^2\) and summing the expected squared entries proves the formula. \(\square\)

The first sum is squared bias: whenever the taper weight differs from one, a nonzero population covariance is attenuated. The second sum is variance: weights below one reduce the contribution of noisy sample covariances. Since the window is zero beyond distance \(k\), entries there contribute no sampling variance to the estimate, but their population covariances remain part of the bias.

Risk Bounds for Bandable Covariance Matrices

A useful formal setting is a class of covariances whose distant entries have small absolute row sums. Suppose, for constants \(C_0,C>0\) and \(\alpha>0\), that $$ \max_i\sum_{j\ne i}|\Sigma_{ij}|\leq C_0,\qquad \max_i\sum_{|i-j|>m}|\Sigma_{ij}|\leq C m^{-\alpha}\quad(m\geq1). $$ The second condition expresses bandability: as the excluded neighborhood grows, the remaining row mass decreases. Also suppose \(\max_i\Sigma_{ii}\leq M\). A direct bound shows both how the width acts and what this particular Bartlett window costs in bias.

For the first inequality, split each row into distances at most \(m\) and greater than \(m\). At a distance \(d\leq m<k\), \(1-(T_k)_{ij}=d/(k+1)\leq m/k\). The absolute row sum of the corresponding covariances is at most \(C_0\). At distances greater than \(m\), the factor \(1-(T_k)_{ij}\) is at most one, and the bandability condition bounds the remaining row sum by \(Cm^{-\alpha}\). Adding these bounds proves the row-sum inequality. The squared Frobenius norm of the bias is at most \(p\) times the square of its largest absolute row sum: each row’s squared-entry sum is no greater than the square of its absolute-entry sum.

For the variance term, \(\Sigma_{ii}\Sigma_{jj}+\Sigma_{ij}^2\leq2M^2\), since positive semidefiniteness gives \(|\Sigma_{ij}|^2\leq\Sigma_{ii}\Sigma_{jj}\leq M^2\). Also, each row of \(T_k\) has at most \(2k+1\) nonzero entries, each at most one, so \(\sum_{i,j}(T_k)_{ij}^2\leq p(2k+1)\leq3pk\). Substitution into the risk identity gives the displayed variance bound.

For sufficiently large \(k\), choosing \(m\) on the order of \(k^{1/(\alpha+1)}\) balances the two bias terms: \(m/k\) and \(m^{-\alpha}\) then have order \(k^{-\alpha/(\alpha+1)}\). The resulting bound has squared-bias order \(p k^{-2\alpha/(\alpha+1)}\) and variance order \(pk/n\). This bound makes the basic decision clear: increasing \(k\) reduces the attenuation bias in this bound while increasing the number of noisy entries retained.

How Tapering Differs from Hard Banding

Hard banding keeps entries within a chosen distance and sets all farther entries to zero. Its abrupt cutoff does not attenuate the entries it retains, so its bias can be smaller than that of the Bartlett taper for a covariance concentrated near the diagonal. Tapering instead downweights progressively more distant entries. Both estimators limit the number of noisy sample covariances used, but only the positive-semidefinite taper has the guarantee proved above. Hard banding does not in general preserve positive semidefiniteness.

Common Pitfall and Practical Interpretation

Common mistake: “The sample covariance is positive semidefinite, so any operation that sets some of its entries to zero must also leave it positive semidefinite.” This reasoning is false: positive semidefiniteness constrains the entries jointly, and hard banding can break those constraints. The correction is to check the transformation itself. Entrywise multiplication preserves positive semidefiniteness when the multiplier matrix is positive semidefinite; the Bartlett window meets that condition.

The guarantee does not make every taper width appropriate, nor does it mean the estimator is unbiased. The risk identity shows exactly what is being traded: weights close to one limit tapering bias, while weights appreciably below one reduce variance. The Bartlett window also attenuates even the nearest off-diagonal entries, so its bias may be consequential when strong local covariances matter. Width should therefore be considered alongside the assumed decay of covariance with distance and the loss used to assess the estimate.