Why the Choice of Observed Cases Matters
A covariance matrix summarizes variances and covariances on a common scale for one multivariate population. With incomplete observations, two simple ways to estimate its entries are to retain only rows observed on every variable, or to use every available pair separately. They use different data, and each has limitations: complete cases can be unrepresentative, while pairwise estimates can fail to describe any coherent multivariate covariance matrix.
The distinction matters even when every individual calculation looks reasonable. A covariance matrix must be positive semidefinite: every linear combination of the variables must have nonnegative variance. Estimates computed from different subsets do not automatically satisfy this requirement. The goal here is to identify what each estimator estimates under stated assumptions and what can go wrong otherwise.
Definitions and Assumptions
Suppose \(X_i=(X_{i1},\ldots,X_{ip})^\mathsf{T}\), \(i=1,\ldots,n\), are independent and identically distributed random vectors with mean \(\mu\) and finite covariance matrix \(\Sigma\). Let \(R_{ij}=1\) if variable \(j\) is observed for case \(i\), and \(R_{ij}=0\) otherwise. A case is complete when all \(p\) variables are observed.
For a variable pair \(j,k\), retain only cases where both entries are observed. Write \(m_{jk}=\sum_i I(R_{ij}=R_{ik}=1)\), and let \(\bar X_{j\mid jk}\) and \(\bar X_{k\mid jk}\) be the two means over that pair's observed cases.
These definitions use a sample denominator, not a population denominator. Each off-diagonal pair can have its own sample size and its own two means. “MCAR” means missing completely at random: the missingness indicators are independent of the full data vector. This is stronger than requiring that each pair's selection be unrelated to its two values.
When Complete Cases Estimate the Target Covariance
Under MCAR, complete-case selection does not favor any particular values of \(X_i\). More precisely, if the complete-case indicator is independent of \(X_i\) and its probability is \(q>0\), then, conditional on the number of complete cases being \(m\), those cases have the original distribution. This gives the usual finite-sample unbiasedness result whenever \(m\geq2\).
Proof. Conditional on \(m_c=m\), the complete vectors \(X_1^*,\ldots,X_m^*\) are iid with mean \(\mu\) and covariance \(\Sigma\). For \(j,k\), the sample covariance entry can be written $$ S^{\mathrm{cc}}_{jk}=\frac{1}{m-1}\left(\sum_{r=1}^m X^*_{rj}X^*_{rk}-m\bar X^*_{j}\bar X^*_{k}\right). $$ We have \(E[\sum_r X^*_{rj}X^*_{rk}]=m(\Sigma_{jk}+\mu_j\mu_k)\). Independence across cases gives \(E[\bar X^*_j\bar X^*_k]=\mu_j\mu_k+\Sigma_{jk}/m\). Substitution yields $$ E[S^{\mathrm{cc}}_{jk}\mid m_c=m] =\frac{m(\Sigma_{jk}+\mu_j\mu_k)-m(\mu_j\mu_k+\Sigma_{jk}/m)}{m-1} =\Sigma_{jk}. $$ This holds for every entry, proving the matrix result. \(\square\)
If \(q>0\), the number of complete cases grows with \(n\) under iid sampling, so complete-case estimates can also be consistent under MCAR and the stated moment conditions. But when \(q\) is small, few complete cases may remain, producing high variability or no estimate at all. Under value-dependent selection, the complete cases instead represent a selected distribution, not necessarily the distribution with covariance \(\Sigma\).
Worked Example: Complete-Case Selection Changes the Covariance
Let the population assign probability \(1/4\) to each of \((-1,-1),(-1,1),(1,-1),(1,1)\). The two coordinates have means zero, variances one, and covariance zero: the positive and negative cross-products cancel.
Now suppose a case is complete exactly when \(X=Y\). The complete cases can only be \((-1,-1)\) and \((1,1)\). In this selected distribution, both means are zero and every cross-product is one, so the covariance is one. Thus the complete-case covariance targets the selected covariance \(1\), not the population covariance \(0\). Here selection depends on the values, so MCAR does not hold.
Pairwise Estimates: Useful Entries, Different Samples
For one fixed pair, the sample covariance formula has the usual unbiasedness property if the selected observations for that pair are an iid sample from the pair's target distribution. In particular, if the indicator that both variables are observed is independent of \((X_{ij},X_{ik})\), then, conditional on \(m_{jk}\geq2\), \(S^{\mathrm{pair}}_{jk}\) is unbiased for \(\Sigma_{jk}\). Under MCAR this condition holds for every pair. Under more general missingness, it must be checked pair by pair; observed pairs need not represent the target pair distribution.
Even when every pair is unbiased under MCAR, the estimates can use different \(m_{jk}\) and different pair-specific means. Consequently, the resulting entries are not necessarily the entries of a positive semidefinite matrix. The next calculations illustrate why it is important to center each pair using its own means and to use its actual denominator.
Worked Example: Auditing Pair-Specific Covariance Calculations
Suppose the four observed \((X,Y)\) pairs are \((7/4,21/16)\), \((7/4,13/16)\), \((7/4,-3/16)\), and \((-9/4,17/16)\). Both sample means are \(3/4\). The centered deviations for \(X\) are \(1,1,1,-3\); those for \(Y\) are \(9/16,1/16,-15/16,5/16\). The four centered cross-products are therefore \(9/16,1/16,-15/16,-15/16\), whose sum is \(-20/16=-5/4\). Since there are four pairs, $$ S^{\mathrm{pair}}_{XY}=\frac{-5/4}{4-1}=-\frac{5}{12}. $$
For a separate pair \((Y,Z)\), suppose the observed pairs are \((0,0),(1,1),(1,2)\). The means are \(2/3\) and \(1\). The centered cross-products are \((-\frac23)(-1)=\frac23\), \((\frac13)(0)=0\), and \((\frac13)(1)=\frac13\); their sum is \(1\). Thus $$ S^{\mathrm{pair}}_{YZ}=\frac{1}{3-1}=\frac12. $$ These calculations use different sample sizes and need not share cases. The pair-specific means and denominators are part of each calculation, not optional details.
Why Pairwise Covariances Can Fail Positive Semidefiniteness
A symmetric matrix \(A\) is positive semidefinite if \(a^\mathsf{T}Aa\geq0\) for every vector \(a\). Every genuine covariance matrix has this property because \(a^\mathsf{T}\Sigma a=\operatorname{Var}(a^\mathsf{T}X)\). A covariance matrix computed from a common set of complete cases also has the property, whether or not the sample is representative of the target population.
Proof. For any vector \(a\in\mathbb R^p\), let \(\bar X_c\) be the complete-case mean. Then $$ a^\mathsf{T}S^{\mathrm{cc}}a =\frac{1}{m_c-1}\sum_{i\text{ complete}} \left\{a^\mathsf{T}(X_i-\bar X_c)\right\}^2\geq0. $$ Every term is a square and the denominator is positive. Hence the inequality holds for every \(a\), which is precisely positive semidefiniteness. \(\square\)
There is no such sum-of-squares representation for a matrix whose entries were computed from different samples. The following explicit construction shows the consequence.
Worked Example: An Available-Case Matrix That Is Not Positive Semidefinite
Let \(a=\sqrt{3}/2\). Take six rows with the indicated observed variables; a dash means missing:
| Rows | X | Y | Z |
|---|---|---|---|
| 1 | \(-a\) | \(-a\) | — |
| 2 | \(a\) | \(a\) | — |
| 3 | \(-a\) | — | \(-a\) |
| 4 | \(a\) | — | \(a\) |
| 5 | — | \(-a\) | \(a\) |
| 6 | — | \(a\) | \(-a\) |
Each variable has four observed values \((-a,a,-a,a)\), mean zero, and sample variance \(4a^2/3=1\). The \(XY\) pair has two observations \((-a,-a),(a,a)\), giving covariance \(2a^2=3/2\). The \(XZ\) pair gives the same covariance. The \(YZ\) pair is \((-a,a),(a,-a)\), giving covariance \(-2a^2=-3/2\). Hence the pairwise matrix is $$ S^{\mathrm{pair}}= \begin{pmatrix} 1&3/2&3/2\\ 3/2&1&-3/2\\ 3/2&-3/2&1 \end{pmatrix}. $$ For \(u=(1,-1,-1)^\mathsf{T}\), direct multiplication gives \(u^\mathsf{T}S^{\mathrm{pair}}u=-6<0\). Thus this symmetric pairwise covariance matrix is not positive semidefinite, despite each entry being a valid sample covariance for its own observed subset.
Practical Interpretation and Common Pitfalls
Complete-case and pairwise estimators answer different operational questions. Complete-case analysis estimates covariance among jointly observed cases; it estimates the target covariance under assumptions such as MCAR, but can lose substantial information. Pairwise calculations retain cases for each entry, and under pairwise MCAR can estimate each target entry without requiring a case to be complete. However, the entries may come from different samples, making the assembled matrix unsuitable for methods that require a valid covariance matrix.
A common mistake is to argue, “Every pairwise covariance is unbiased, so the assembled covariance matrix must be valid.” Unbiasedness is an entry-by-entry expectation statement; positive semidefiniteness is a joint constraint on all entries of a realized matrix. The six-row example demonstrates that the latter does not follow from the former.
- Do not assume complete cases are representative. Under value-dependent missingness, their covariance can differ from the target covariance.
- Do not silently combine samples. Report \(m_{jk}\) for each pair when using available cases.
- Do not overlook undefined entries. A pair with fewer than two observed joint cases has no sample covariance under this definition.
- Do not treat symmetry as validity. A symmetric matrix can still have a negative quadratic form and fail to be positive semidefinite.
When a downstream procedure requires a coherent multivariate covariance estimate, an entrywise pairwise matrix should be checked rather than presumed valid. The selection assumptions, number of observations per pair, and matrix properties all matter; no single simple deletion rule resolves every incomplete-data problem.
Check Your Understanding
Answer each question by stating the relevant assumption or calculation.
- Under what condition is the complete-case sample covariance unbiased for the population covariance, conditional on at least two complete observations?
- Why can complete-case covariance be biased when completeness depends on the observed values?
- For a pair with \(m_{jk}=5\), which observations determine its two means and what denominator appears in the sample covariance?
- Explain why unbiasedness of every pairwise covariance entry does not imply that the assembled matrix is positive semidefinite.
- Give the sum-of-squares argument showing that a complete-case sample covariance matrix is positive semidefinite.