Why MAR Can Bias Complete-Case Estimates

Why MAR Does Not Guarantee an Unbiased Complete-Case Estimate

Suppose an outcome is missing more often for one age group than another, and age is recorded for everyone. This response pattern can be missing at random (MAR): after conditioning on the observed age, whether the outcome is observed does not depend on the outcome itself. Yet the complete cases can contain a different mix of age groups from the full sample. If average outcome values differ by age, that changed mix can alter a complete-case estimate of the overall mean.

The key distinction is between a missingness mechanism and the distribution of the people or units retained for analysis. MAR describes how missingness relates to the data; it does not say that complete cases represent the full sample for every quantity of interest.

Definition: Consider a partially observed outcome $Y$ and a fully observed covariate $X$. Let $R=1$ if $Y$ is observed and $R=0$ if it is missing. In this setting, MAR means that the probability of observing $Y$, conditional on the observed covariate, does not depend further on the value of $Y$: $$ P(R=1\mid Y,X)=P(R=1\mid X). $$ A complete-case estimate uses only records with $R=1$. Here, “complete case” refers to having the outcome and covariate needed for the analysis; because $X$ is fully observed, this means $Y$ is observed.

The definition allows response rates to differ across values of $X$. It does not require equal response rates, nor does it require that the distribution of $X$ among respondents match its distribution in the full sample. The examples below use a fully observed covariate so that this composition change is easy to see.

How Selection Changes the Covariate Mix

Write $\pi(x)=P(R=1\mid X=x)$ for the response probability at covariate value $x$, and $m(x)=E(Y\mid X=x)$ for the conditional mean outcome. Under MAR, the distribution of $Y$ within each $X$ group is unchanged by selecting respondents: $E(Y\mid X=x,R=1)=m(x)$, whenever $\pi(x)>0$. But the proportions of the $X$ groups among respondents can change.

The following identity shows why this matters for the overall mean. It assumes the expectations exist and that the response probability is positive for the covariate values represented in the target population.

E(Y\mid R=1) =\frac{E(YR)}{P(R=1)} =\frac{E\{m(X)\pi(X)\}}{E\{\pi(X)\}} =E\{m(X)\}+\frac{\operatorname{Cov}\{m(X),\pi(X)\}}{E\{\pi(X)\}}.

To justify the second equality, condition on $X$. MAR gives $E(YR\mid X)=E(Y\mid X,R=1)P(R=1\mid X)=m(X)\pi(X)$. Also, $E(R\mid X)=\pi(X)$, so $P(R=1)=E\{\pi(X)\}$. The last equality follows from the definition of covariance. Since $E(Y)=E\{m(X)\}$, the difference between the complete-case mean and the full-sample mean is

E(Y\mid R=1)-E(Y) =\frac{\operatorname{Cov}\{m(X),\pi(X)\}}{E\{\pi(X)\}}.

Thus, unequal response probabilities can bias the complete-case mean when they are associated with the conditional outcome means. If groups with higher mean outcomes have lower response probabilities, the covariance is negative and the complete-case mean tends to be lower. If those groups have higher response probabilities, it tends to be higher. This is a statement about a marginal mean; it does not imply that every estimate from every complete-case analysis is biased.

Worked Example: A Lower Complete-Case Mean Under MAR

Consider a hypothetical sample of 1,000 people: 500 in a younger group and 500 in an older group. Let $Y$ be a clinical score measured in points. Suppose the full-sample mean score is 40 points in the younger group and 60 points in the older group. The outcome is observed for 90% of the younger group and 50% of the older group. Assume response is unrelated to $Y$ within each age group, as required for MAR in this example.

The full-sample mean is the equally weighted average of the two group means: $$ \bar{Y}_{\text{full}}=\frac{500(40)+500(60)}{1000} =\frac{50{,}000}{1000}=50.00\text{ points}. $$

The numbers of respondents are $500(0.90)=450$ younger people and $500(0.50)=250$ older people. Using their group means, the complete-case mean is $$ \bar{Y}_{\text{CC}}=\frac{450(40)+250(60)}{450+250} =\frac{33{,}000}{700}=47.14\text{ points}. $$

The complete-case estimate is about $2.86$ points below the full-sample mean. The reason is not that response depends on the score within age groups. It is that older people, whose mean score is higher, make up $250/700=35.71\%$ of the respondents but $500/1000=50.00\%$ of the full sample. In this hypothetical example, MAR holds, but the unadjusted complete-case estimate does not recover the full-sample mean.

A Proportion Can Be Affected in the Same Way

The same composition argument applies to proportions. For a binary outcome, the mean of its zero-one indicator is the outcome prevalence. If that prevalence differs across covariate groups and response rates also differ across those groups, the overall prevalence among complete cases can differ from the full-sample prevalence even when response is MAR.

Worked Example: A Complete-Case Prevalence

In a hypothetical sample of 2,000 people, 1,000 are in each of two age groups. A binary outcome occurs in 20% of the younger group and 60% of the older group. The outcome is observed for 90% of younger people and 50% of older people. Assume response does not depend on the outcome within age group.

The full-sample prevalence is $$ \hat{p}_{\text{full}} =\frac{1000(0.20)+1000(0.60)}{2000} =\frac{200+600}{2000} =0.4000. $$

There are 900 younger respondents, of whom $900(0.20)=180$ have the outcome, and 500 older respondents, of whom $500(0.60)=300$ have it. The complete-case prevalence is therefore $$ \hat{p}_{\text{CC}} =\frac{180+300}{900+500} =\frac{480}{1400} =0.3429. $$

The complete-case prevalence is 34.29%, compared with 40.00% in the full sample, a difference of $-5.71$ percentage points. The calculation shows a marginal difference; it does not show that the age-specific prevalences differ between respondents and nonrespondents. Under the stated within-group response assumption, the age-specific prevalences are the same, but the age composition is not.

Correcting the Composition in a Simple Setting

When the full-sample distribution of $X$ is known and there are respondents at every relevant value of $X$, one simple way to target a marginal mean is to combine the observed within-$X$ means using the full-sample proportions of $X$, rather than the respondent proportions. This is called standardization. It relies on MAR so that the observed outcome distribution within each $X$ group represents that group’s outcome distribution in the full sample.

Worked Example: Standardizing the Mean to the Full Sample

Use the hypothetical clinical-score data from the first example. The full sample is 50% younger and 50% older, and the respondent means within those groups are 40 and 60 points. Standardizing to the full-sample age distribution gives $$ \bar{Y}_{\text{standardized}} =0.50(40)+0.50(60) =20+30=50.00\text{ points}. $$

This matches the stipulated full-sample mean. Equivalently, if the response probabilities are known, weight each respondent by the reciprocal of the response probability in their group. The weighted total is $450(40)/0.90+250(60)/0.50=20{,}000+30{,}000=50{,}000$, and the weighted count is $450/0.90+250/0.50=500+500=1{,}000$. Their ratio is $50{,}000/1{,}000=50.00$ points. In practice, response probabilities are usually estimated rather than known, and the validity of the correction depends on the covariates and model used.

These calculations illustrate the role of assumptions, not a universal repair. Standardization needs the relevant covariate distribution and reliable within-covariate outcome information. Inverse-probability weighting additionally needs suitable response probabilities. If a group has no respondents, its outcome distribution cannot be recovered from these data by simply reweighting observed outcomes. This is a positivity failure: the needed response probability is zero for a group in the target population.

When MAR Does—and Does Not—Create This Bias

MAR makes a complete-case estimate of the marginal mean vulnerable to bias, but does not guarantee that bias occurs. In the identity above, the complete-case mean equals the full-sample mean if $\operatorname{Cov}\{m(X),\pi(X)\}=0$. For example, if the conditional mean $m(X)$ is constant across $X$, then differences in response rates do not change the mean. Equality can also occur if the response probabilities and conditional means vary in ways that yield zero covariance. Such equality for one particular estimate does not establish MCAR or show that other estimates are unbiased.

The target of estimation matters too. A complete-case estimate of an overall mean combines groups using the respondent distribution of $X$. A model for the conditional relationship between $Y$ and $X$ asks a different question. In some settings, if response depends only on fully observed predictors and the conditional outcome model is correctly specified, estimating that conditional model from respondents can preserve its parameters. That does not make the respondent sample representative for a marginal mean, nor does it guarantee validity under model misspecification or when other variables used in the analysis are missing.

Diagnostics, Alternatives, and Limitations

Useful checks begin with the observed data. Compare response rates across relevant observed covariates, and compare the covariate distributions in the complete cases and the full sample. A marked difference in those distributions makes the composition issue visible. If the outcome is observed for some people in each group, also compare observed outcome summaries across groups; these summaries help assess whether the covariate is related to the outcome among respondents.

These checks do not verify MAR. Because $Y$ is unavailable when $R=0$, observed data generally cannot establish whether response depends on the unobserved value of $Y$ after conditioning on $X$. Also, a covariate omitted from the response or outcome model can leave important differences unaccounted for. The more relevant predictors of response and outcome that are observed and included, the more plausible the MAR analysis may be, but plausibility is not proof.

The main alternatives answer different needs. Under MCAR, complete cases are not selected according to the observed or unobserved data, so their covariate composition is representative in expectation, subject to sampling variation. Under MAR, unadjusted complete-case means and proportions can be biased as shown above. Standardization or inverse-probability weighting can address observed composition differences when their assumptions and data support them. Multiple imputation by chained equations is another approach for analyses with missing values; it uses a model to represent uncertainty about those values under its assumptions. No method can recover information for a group with no observed outcomes without additional assumptions or external information.

A common mistake is to reason, “The outcome is missing at random, so the observed outcomes are a random sample and their average is unbiased.” That conclusion confuses MAR with MCAR. Under MAR, respondents may be selected by observed covariates. The correction is to ask whether the covariate distribution differs between complete cases and the target sample and whether outcome means or risks vary across that covariate. If both vary together, the complete-case marginal estimate can be biased.

Key takeaway: Under MAR, outcome distributions can be comparable between respondents and nonrespondents within levels of observed covariates, while complete cases still have the wrong covariate mix for a marginal estimate. MAR permits complete-case bias; it does not imply that every complete-case estimate must be biased.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

If response depends on age but not on the outcome within age groups, is the response pattern compatible with MAR in the one-outcome setting described here?
A covariate has response probabilities of 0.80 in both groups. What does this imply about the response-related change in the two-group covariate mix?
Suppose the full sample is 40% in group A and 60% in group B. The group-specific outcome means are 10 and 20 units, and response rates are 80% and 40%. Calculate the full-sample mean and the complete-case mean.
Can observing different response rates across covariate groups establish that missingness is MAR rather than MNAR?

Solutions. 1. Yes. The defining condition is that response probability does not depend on the unobserved outcome after conditioning on the observed age; response may vary with age. 2. Equal response probabilities do not shift the relative group proportions among respondents, because both groups are retained at the same rate. 3. The full-sample mean is $0.40(10)+0.60(20)=16.00$ units. Expected respondent shares are proportional to $0.40(0.80)=0.32$ and $0.60(0.40)=0.24$, so the complete-case mean is $\{0.32(10)+0.24(20)\}/(0.32+0.24)=8.00/0.56=14.29$ units. 4. No. Those rates describe relationships with observed covariates, but cannot show whether response also depends on the unobserved outcome after conditioning on them.