Estimating Equations Under MCAR

Why MCAR Gives More Than One Valid Equation

With incomplete multivariate observations, a single case may contribute information about some coordinates but not others. Under missing completely at random (MCAR), the observation pattern is independent of the data vector. That independence makes it possible to construct unbiased moment equations both from cases that are entirely observed and from the components and pairs that are observed. These choices use different amounts of data and need not produce the same finite-sample estimates.

The focus here is the estimating equations themselves: what they are, why their expectations vanish at the population parameters, and what conditions make their roots consistent. An equation with expectation zero is not automatically an unbiased estimator. In particular, solving an equation that uses an estimated mean can introduce finite-sample bias into a covariance estimate.

Notation and MCAR Conditions

Let \(X_i=(X_{i1},\ldots,X_{ip})^\mathsf{T}\), for \(i=1,\ldots,n\), be independent and identically distributed random vectors with mean \(\mu\) and covariance matrix \(\Sigma\). Assume each coordinate has a finite second moment. For case \(i\), let \(R_{ij}=1\) if coordinate \(j\) is observed and \(R_{ij}=0\) otherwise. The vector \(R_i=(R_{i1},\ldots,R_{ip})\) records the observation pattern.

Definition: The data are MCAR when \(R_i\) is independent of \(X_i\). For a coordinate \(j\), write \(q_j=P(R_{ij}=1)\). For a pair \(j,k\), write \(q_{jk}=P(R_{ij}=1,R_{ik}=1)\). Write \(q_C=P(R_{i1}=\cdots=R_{ip}=1)\) for the probability that a case is complete. For estimating a particular mean or covariance component, the corresponding observation probability must be positive.

The MCAR condition does not require the coordinates of \(R_i\) to be mutually independent. For example, two variables may tend to be observed together. What matters for the equations below is that the relevant observation indicator is independent of the data values. The earlier material on MCAR establishes why complete cases represent the same data distribution as the full population under this condition.

Complete-Case Equations

Let \(C_i=\prod_{j=1}^p R_{ij}\), so \(C_i=1\) exactly when case \(i\) is complete. A complete-case mean equation and a complete-case covariance equation are

\sum_{i=1}^n C_i(X_i-\mu)=0, \qquad \sum_{i=1}^n C_i\{(X_i-\mu)(X_i-\mu)^\mathsf{T}-\Sigma\}=0.

The equations are understood component by component. When there is at least one complete case, their joint roots are the complete-case sample mean and the covariance moment matrix centered at that mean, with divisor equal to the number of complete cases. The covariance equation uses the same complete cases for every entry, so its root is positive semidefinite.

Result: If \(q_C>0\), both complete-case estimating functions have expectation zero at the true \((\mu,\Sigma)\). In fact, independence gives \(E[C_i(X_i-\mu)]=E[C_i]E[X_i-\mu]=0\), and likewise \(E[C_i\{(X_i-\mu)(X_i-\mu)^\mathsf{T}-\Sigma\}]=0\).

The result validates the population equations; it does not guarantee that a particular sample contains any complete cases. When \(q_C\) is small, complete-case equations may be statistically inefficient or practically unusable even though they are valid under MCAR.

Equations Using All Observed Components

A component-wise mean equation uses every observed value of coordinate \(j\), whether or not the other coordinates in that case are observed:

\sum_{i=1}^n R_{ij}(X_{ij}-\mu_j)=0,\qquad j=1,\ldots,p.

For covariance, coordinate \(j\)'s variance equation uses all cases where \(j\) is observed. An off-diagonal covariance equation uses all cases where both variables are observed:

\sum_{i=1}^n R_{ij}R_{ik} \left\{(X_{ij}-\mu_j)(X_{ik}-\mu_k)-\Sigma_{jk}\right\}=0, \qquad 1\leq j\leq k\leq p.

For \(j=k\), the product \(R_{ij}R_{ik}\) is \(R_{ij}\). For \(j\ne k\), the equation uses only jointly observed pairs. At the population parameter, MCAR and the finite second moments imply

E\!\left[R_{ij}R_{ik} \left\{(X_{ij}-\mu_j)(X_{ik}-\mu_k)-\Sigma_{jk}\right\}\right] =q_{jk}\left(E[(X_{ij}-\mu_j)(X_{ik}-\mu_k)]-\Sigma_{jk}\right)=0.

When the equations have roots, the mean roots are coordinate-specific observed means. The covariance roots are pair-specific observed cross-products centered at the estimated coordinate means. This is a set of moment equations, not a promise that all roots together form a valid covariance matrix.

Proof of Unbiasedness and Consistency

The following result states precisely what MCAR provides. For consistency, assume observations are independent and identically distributed across cases, the relevant observation probability is positive, and the stated second moments exist. For covariance entries, these conditions ensure that the products in the equations are integrable.

Theorem: Under MCAR, the complete-case equations have expectation zero at \((\mu,\Sigma)\) when \(q_C>0\). The component-wise mean and pairwise covariance equations have expectation zero at \((\mu,\Sigma)\) when their respective \(q_j\) and \(q_{jk}\) are positive. The roots of the component-wise equations are consistent for each estimable mean and covariance entry; the complete-case roots are consistent when \(q_C>0\).

Proof. Independence of \(R_i\) and \(X_i\) gives

E[C_i(X_i-\mu)]=q_C E[X_i-\mu]=0

and

E[C_i\{(X_i-\mu)(X_i-\mu)^\mathsf{T}-\Sigma\}] =q_C\{E[(X_i-\mu)(X_i-\mu)^\mathsf{T}]-\Sigma\}=0.

For a component mean, the same calculation gives \(E[R_{ij}(X_{ij}-\mu_j)]=q_jE[X_{ij}-\mu_j]=0\). For a covariance entry, independence gives the displayed pairwise expectation above, which is zero because \(E[(X_{ij}-\mu_j)(X_{ik}-\mu_k)]=\Sigma_{jk}\). This proves the expectation claims.

For consistency of a component mean root, its equation has the solution

\hat{\mu}_j= \frac{n^{-1}\sum_{i=1}^n R_{ij}X_{ij}} {n^{-1}\sum_{i=1}^n R_{ij}}.

By the law of large numbers, the numerator converges in probability to \(E[R_{ij}X_{ij}]=q_j\mu_j\), and the denominator converges in probability to \(q_j>0\). The ratio therefore converges in probability to \(\mu_j\). The complete-case mean has the same argument with \(C_i\) in place of \(R_{ij}\).

For a covariance entry, the root can be written as the observed-pair average of \((X_{ij}-\hat{\mu}_j)(X_{ik}-\hat{\mu}_k)\). The law of large numbers applied to the observed-pair count and to the product centered at the true means gives a limiting denominator \(q_{jk}\) and numerator \(q_{jk}\Sigma_{jk}\). Replacing the true means by their consistent estimates changes the normalized average by terms that converge in probability to zero: each term is a product of a mean-estimation error and an observed-pair average of a coordinate deviation, or a product of two mean-estimation errors. Those averages are bounded in probability by the law of large numbers and finite second moments. Thus the root converges in probability to \(\Sigma_{jk}\). The complete-case covariance follows by the same argument using \(C_i\), which has positive expectation \(q_C\). \(\square\)

This consistency argument concerns roots of moment equations. It does not assert a particular asymptotic variance or optimal weighting; those depend on how the equations are combined and on their variability.

Worked Comparisons

Worked Example: How Observation Probabilities Enter the Equations

Suppose \(X\) is bivariate with mean \((2,-1)^\mathsf{T}\) and covariance \(\begin{pmatrix}4&1.5\\1.5&9\end{pmatrix}\). Independently of \(X\), a case is complete with probability \(0.5\), has only coordinate 1 observed with probability \(0.2\), has only coordinate 2 observed with probability \(0.2\), or has neither observed with probability \(0.1\). Find the expectations of the relevant one-case estimating functions at the true parameters.

The marginal observation probabilities are \(q_1=0.5+0.2=0.7\) and \(q_2=0.5+0.2=0.7\). The joint observation probability is \(q_{12}=0.5\). Thus the expected complete-case mean equation is \(0.5E[X-\mu]=0\), and the complete-case covariance equation has expectation \(0.5(\Sigma-\Sigma)=0\).

For the component-wise mean equations, the expectations are \(0.7E[X_1-2]=0\) and \(0.7E[X_2+1]=0\). For the cross-covariance equation, the centered product has expectation \(E[(X_1-2)(X_2+1)]=1.5\), so its expected estimating function is \(0.5(1.5-1.5)=0\). For the two variance equations, the corresponding expectations are \(0.7(4-4)=0\) and \(0.7(9-9)=0\). The different probabilities scale the equations, but do not change their population roots.

Worked Example: Solving Complete-Case and Component Mean Equations

Six bivariate cases have the observed values \((1,2)\), \((3,4)\), \((5,\text{missing})\), \((\text{missing},8)\), \((7,10)\), and \((\text{missing},12)\). Solve the complete-case and component-wise mean equations.

There are three complete cases. The complete-case equations give \(\hat{\mu}_{1,C}=(1+3+7)/3=11/3\) and \(\hat{\mu}_{2,C}=(2+4+10)/3=16/3\).

There are four observed values of coordinate 1 and five of coordinate 2. The component-wise equations give \(\hat{\mu}_1=(1+3+5+7)/4=4\) and \(\hat{\mu}_2=(2+4+8+10+12)/5=36/5=7.2\).

The two answers differ because the equations average over different sets of cases. Under MCAR, both are roots of valid equations for their respective sampling schemes; this particular small data set does not itself establish MCAR, and the numerical difference is not evidence that either population equation is biased.

Worked Example: Solving a Pairwise Covariance Equation

Use the same six cases and the component-wise mean roots \(\hat{\mu}_1=4\), \(\hat{\mu}_2=7.2\). Only three cases have both coordinates observed. Solve the pairwise cross-covariance equation, using divisor equal to the number of jointly observed pairs.

For \((1,2)\), the centered product is \((1-4)(2-7.2)=(-3)(-5.2)=15.6\). For \((3,4)\), it is \((3-4)(4-7.2)=(-1)(-3.2)=3.2\). For \((7,10)\), it is \((7-4)(10-7.2)=(3)(2.8)=8.4\). Their sum is \(15.6+3.2+8.4=27.2\), so the equation root is \(\hat{\Sigma}_{12}=27.2/3=9.066\overline{6}\).

The calculation uses all jointly observed pairs, not only complete cases in a higher-dimensional setting. It also illustrates that the mean equations and covariance equation can use different numbers of observations. In a finite sample, the result is a moment root; unbiasedness of the estimating function does not mean this root is exactly unbiased.

What the Comparison Means

Complete-case equations provide a coherent vector-level analysis: every mean and covariance entry is computed from the same subset, and the resulting covariance moment matrix is positive semidefinite. Their limitation is that they discard a case even when some of its coordinates are observed. Component-wise equations retain those values: each mean uses every observation of its coordinate, and each covariance entry uses every observed pair for that entry. Under MCAR and positive observation probabilities, these are also unbiased equations and have consistent roots.

The equations need not have equal precision. If \(q_C\) is much smaller than the marginal and pairwise observation probabilities, complete-case equations use substantially fewer observations for some parameters. On the other hand, component-wise equations use different subsets for different entries. Their separate roots are not automatically constrained to assemble into a positive semidefinite covariance matrix. A covariance model or an estimation procedure may impose additional structure, but that is a further choice beyond unbiasedness of each moment equation.

Common Mistake

Wrong reasoning: “The estimating function has expectation zero, so its root is an unbiased estimator.” The expectation-zero property applies to the function evaluated at the true parameter. Once the equation is solved, its root can involve random denominators and estimated centering values.
Correction: Call the equation unbiased when its expectation is zero at the target parameter. Establish unbiasedness of its root separately if that is needed. Consistency, proved above under the stated conditions, is a different property from finite-sample unbiasedness.
Wrong reasoning: “MCAR means every observation probability is the same.” MCAR means the pattern is independent of the data; marginal and pairwise observation probabilities can differ.
Correction: Use the probability for the equation at hand: \(q_j\) for mean \(j\), \(q_{jk}\) for covariance entry \(jk\), or \(q_C\) for complete-case equations.

Key takeaway: Under MCAR, independence of the observation indicators and the data makes complete-case and component-wise moment equations unbiased at the true mean and covariance. Positive observation probabilities support consistency, but equation validity alone does not guarantee finite-sample unbiasedness, equal efficiency, or a positive semidefinite matrix from separately solved covariance equations.

Check Your Understanding

Use the equations and assumptions developed above to answer these questions.

Why does \(E[R_{ij}(X_{ij}-\mu_j)]=0\) under MCAR?
Which observation probability must be positive for the pairwise equation for \(\Sigma_{jk}\) to identify its moment root?
Explain why a complete-case covariance equation can be valid under MCAR yet use far fewer observations than component-wise equations.
In what sense is an estimating equation unbiased, and why does that not by itself prove its root is unbiased?
Why can covariance equations based on different observed pairs fail to produce a positive semidefinite matrix when solved entry by entry?