Why Covariance-Matrix Consistency Reduces to Moment Convergence
A covariance matrix summarizes the variances of several variables and the covariances between them. In econometric applications, those entries often serve as inputs to estimates and standard errors. A basic asymptotic question is whether the sample covariance matrix approaches the population covariance matrix as the sample grows. The key is to express the sample covariance in terms of empirical first and second cross-moments, then apply convergence results to those moments.
This argument does not require a separate limit calculation for every centered product. Once the sample means and raw second cross-moments converge, the sample covariance follows by subtraction and multiplication. The result applies to a vector with any fixed finite number of components; a bivariate case makes the entries easy to inspect.
The prime denotes transpose, so each outer product \((X_i-\bar X_n)(X_i-\bar X_n)'\) is a \(k\)-by-\(k\) matrix. Its \(j,\ell\) entry is the product of the centered \(j\)th and \(\ell\)th observations. The diagonal entries of \(\Sigma\) are variances; its off-diagonal entries are covariances.
The Raw-Moment Identity
Define the empirical raw second cross-moment matrix \(Q_n=n^{-1}\sum_{i=1}^n X_iX_i'\). Expanding the centered products gives an exact finite-sample identity:
To see why, expand each summand as \(X_iX_i'-X_i\bar X_n'-\bar X_nX_i'+\bar X_n\bar X_n'\). After averaging, the second and third terms each equal \(\bar X_n\bar X_n'\), and the fourth term also equals \(\bar X_n\bar X_n'\). Combining them leaves \(Q_n-\bar X_n\bar X_n'\). Thus centering the observations subtracts the outer product of their sample mean from their empirical raw second moments.
The identity explains the theorem: convergence of \(\bar X_n\) implies convergence of each entry of \(\bar X_n\bar X_n'\) to the corresponding entry of \(\mu\mu'\). Subtracting this limit from the limit \(M\) gives the covariance matrix. In fixed finite dimension, entrywise convergence is equivalent to convergence in any matrix norm, so the statement also gives convergence of the matrix as a whole.
Conditions That Deliver the Moment Limits
A standard sufficient set of conditions is that \(X_1,X_2,\ldots\) are independent and identically distributed random vectors and every component has a finite second moment: \(E[X_{ij}^2]<\infty\) for each \(j\). The weak law of large numbers then gives convergence in probability of each component of \(\bar X_n\) to its mean, and of each entry of \(Q_n\) to its expected value.
For the second statement, the entry in position \(j,\ell\) is the sample average of \(X_{ij}X_{i\ell}\). It has a finite absolute expectation because
Thus the finite-mean weak law applies to these products. Independence and identical distribution, along with the stated moments, are sufficient conditions—not the only possible route. More generally, the theorem applies whenever the first and second empirical moments have the stated probability limits, whatever assumptions establish those limits.
Worked Example: Deriving the Limit From Population Moments
Suppose hypothetical independent, identically distributed bivariate observations have mean \(\mu=(2,-1)'\) and covariance matrix $$ \Sigma=\begin{pmatrix}4&1.5\\1.5&9\end{pmatrix}. $$ Assume the component second moments are finite. Find the limit of the sample covariance matrix.
The raw second-moment matrix satisfies \(M=E[X_iX_i']=\Sigma+\mu\mu'\). The mean outer product is $$ \mu\mu'=\begin{pmatrix}2\\-1\end{pmatrix}\begin{pmatrix}2&-1\end{pmatrix} =\begin{pmatrix}4&-2\\-2&1\end{pmatrix}. $$ Therefore $$ M=\begin{pmatrix}4&1.5\\1.5&9\end{pmatrix} +\begin{pmatrix}4&-2\\-2&1\end{pmatrix} =\begin{pmatrix}8&-0.5\\-0.5&10\end{pmatrix}. $$
The finite second moments justify convergence of the empirical first and second moments by the weak law. Hence \(Q_n\to_p M\) and \(\bar X_n\to_p\mu\). Substituting their limits into the raw-moment identity gives $$ S_n\to_p \begin{pmatrix}8&-0.5\\-0.5&10\end{pmatrix} -\begin{pmatrix}4&-2\\-2&1\end{pmatrix} =\begin{pmatrix}4&1.5\\1.5&9\end{pmatrix}. $$ The diagonal entries converge to variances in the squared units of their respective variables; the off-diagonal entries converge to their covariance in the product of the two variables’ units.
Complete Justification, Including the Divisor
The matrix result follows from familiar probability-limit rules applied to a finite collection of entries. In detail, convergence of \(\bar X_n\) gives convergence of each component, and multiplication of convergent components gives convergence of every entry of \(\bar X_n\bar X_n'\). Combining that with convergence of \(Q_n\) and the exact identity yields the limit of each entry of \(S_n\). The covariance identity \(M-\mu\mu'=\Sigma\) follows by expanding \(E[(X_i-\mu)(X_i-\mu)']\).
The divisor \(n-1\) does not change the probability limit. The two versions are related exactly by $$ S_n^{\,*}=\frac{n}{n-1}S_n. $$ Since \(n/(n-1)\to 1\), convergence of \(S_n\) to the finite matrix \(\Sigma\) implies convergence of \(S_n^{\,*}\) to the same matrix. The divisor choice matters for finite samples, but not for consistency as \(n\) grows.
These steps can be organized as follows.
Show that \(\bar X_n\to_p\mu\), using conditions such as the iid assumption and finite component second moments.
Show that each entry of \(Q_n\), an average of \(X_{ij}X_{i\ell}\), converges in probability to \(E[X_{ij}X_{i\ell}]\).
Use \(S_n=Q_n-\bar X_n\bar X_n'\) to obtain the limit \(M-\mu\mu'=\Sigma\).
Use \(S_n^{\,*}=[n/(n-1)]S_n\); the prefactor tends to one.
Worked Example: Computing a Bivariate Sample Covariance
Consider the following hypothetical observations, where the first variable is measured in hours and the second in points:
| Observation | \(X_{i1}\) (hours) | \(X_{i2}\) (points) |
|---|---|---|
| 1 | 1 | 2 |
| 2 | 2 | 1 |
| 3 | 3 | 4 |
| 4 | 4 | 3 |
The sample means are \(\bar X_{n,1}=(1+2+3+4)/4=2.5\) hours and \(\bar X_{n,2}=(2+1+4+3)/4=2.5\) points. The centered observations are \((-1.5,-0.5)\), \((-0.5,-1.5)\), \((0.5,1.5)\), and \((1.5,0.5)\).
The sum of squared deviations for each variable is \(2.25+0.25+0.25+2.25=5\). The sum of cross-products is \(0.75+0.75+0.75+0.75=3\). Using divisor \(n=4\), $$ S_n=\begin{pmatrix}5/4&3/4\\3/4&5/4\end{pmatrix} =\begin{pmatrix}1.25&0.75\\0.75&1.25\end{pmatrix}. $$ The diagonal entries are in hours squared and points squared; each off-diagonal entry is in hours-times-points. With divisor \(n-1=3\), the result is $$ S_n^{\,*}=\begin{pmatrix}5/3&1\\1&5/3\end{pmatrix} \approx\begin{pmatrix}1.667&1\\1&1.667\end{pmatrix}. $$ This finite dataset illustrates the two calculations and their different divisors. It does not, by itself, establish consistency; consistency is a statement about a sequence of sample matrices as the sample size grows under appropriate conditions.
Worked Example: Using Empirical-Moment Limits Directly
For a hypothetical bivariate sequence, suppose the empirical first-moment vector converges in probability to \((3,2)'\), and the empirical raw second-moment matrix converges in probability to $$ Q=\begin{pmatrix}13&7\\7&8\end{pmatrix}. $$ Find the implied probability limit of the sample covariance matrix.
The limiting outer product of the sample mean is $$ \begin{pmatrix}3\\2\end{pmatrix}\begin{pmatrix}3&2\end{pmatrix} =\begin{pmatrix}9&6\\6&4\end{pmatrix}. $$ Subtracting this from the raw second-moment limit gives $$ S_n\to_p \begin{pmatrix}13&7\\7&8\end{pmatrix} -\begin{pmatrix}9&6\\6&4\end{pmatrix} =\begin{pmatrix}4&1\\1&4\end{pmatrix}. $$ Thus the first variable’s variance limit is 4, the second variable’s variance limit is 4, and their covariance limit is 1, with units determined by the variables. This application uses the moment limits directly; a particular data-generating assumption is needed only if one must justify those limits from more primitive conditions.
Common Mistake: Centering Does Not Disappear Without a Limit
A tempting but incomplete argument is: “The sample covariance is an average of products, so the law of large numbers makes it converge to the covariance.” The summands in the sample covariance are centered using the random sample mean, not the fixed population mean. The raw average \(Q_n\) converges to \(E[X_iX_i']\), which is generally not \(\Sigma\). The correction \(\bar X_n\bar X_n'\) is essential: it converges to \(\mu\mu'\), and subtracting it gives \(E[X_iX_i']-\mu\mu'=\Sigma\).
Another limitation is that convergence of empirical first moments alone cannot establish covariance consistency. It says nothing by itself about whether the empirical second cross-moments converge or are finite. Under iid sampling, finite second moments provide a simple sufficient condition for both sets of limits. If those conditions are not available, the moment convergence claims must be justified by other appropriate assumptions rather than presumed.
The practical value of the identity is that it separates the work into two transparent tasks: establish limits for empirical first and second moments, then combine them. It also makes clear why the sample covariance matrix estimates variation around the population mean rather than raw second moments around zero.
Check Your Understanding
Answer each question using the definitions and results above.
- Write the raw-moment identity for \(S_n\), and explain why its limiting value is the population covariance matrix.
- For iid bivariate observations, why do finite second moments ensure finite absolute expectations for the cross-products?
- If \(Q_n\to_p\begin{pmatrix}10&4\\4&6\end{pmatrix}\) and \(\bar X_n\to_p(2,1)'\), what is the probability limit of \(S_n\)?
- Why do the divisors \(n\) and \(n-1\) give covariance matrices with the same probability limit?
- Does convergence of \(\bar X_n\) alone establish consistency of the sample covariance matrix? Explain what additional convergence is needed.