Why Observed Data Cannot Verify MAR

Why MAR Is Not Directly Checkable

When a variable is missing, its value is unavailable precisely for the cases that would be needed to compare it with response status. This creates a basic limit on what a data set can tell us about its own missingness mechanism. Observed response rates and associations are useful evidence, but they cannot generally establish that response is independent of the unobserved values after conditioning on the observed data.

Let $Y=(Y_1,\ldots,Y_p)$ denote the variables of interest, and let $R$ denote the response pattern. For a given pattern $r$, write $Y_{\mathrm{obs}}(r)$ for the components observed under that pattern and $Y_{\mathrm{mis}}(r)$ for the components missing under it. Fully observed covariates can be included among the conditioning information.

Definition: The missing at random (MAR) condition holds when, for each response pattern $r$, the probability of that pattern depends on the data only through the values observed under that pattern: $$ P(R=r\mid Y_{\mathrm{obs}}(r),Y_{\mathrm{mis}}(r)) = P(R=r\mid Y_{\mathrm{obs}}(r)). $$ In words, once the observed values are accounted for, the probability of a response pattern does not additionally depend on the values missing under that pattern. MAR is a condition on the data-generating mechanism, not a pattern that can be read directly from a table.

For one incomplete outcome $Y$ and fully observed covariates $X$, let $R=1$ indicate that $Y$ is observed and $R=0$ that it is missing. The relevant MAR condition is $P(R=1\mid Y,X)=P(R=1\mid X)$, with the corresponding equality for $R=0$. The difficulty is that $Y$ is recorded only when $R=1$. For cases with $R=0$, we cannot compare their response probabilities across their actual, unobserved values of $Y$.

This does not make all analysis of missingness unhelpful. We can compare response rates across observed groups, check whether response is associated with observed variables, and assess whether a proposed MAR explanation is plausible. Such checks address relationships visible in the data. They do not reveal the values of $Y$ among nonrespondents, so they cannot verify the defining conditional independence for those values.

What the Observed-Data Distribution Leaves Out

For a single incomplete outcome, the observed-data probabilities include the chance of being missing and, among observed cases, the distribution of the recorded outcomes. They do not separately reveal the distribution of outcomes among missing cases. The distinction can be expressed by integrating over the unavailable values:

p(r,y_{\mathrm{obs}}) = \int p(r\mid y_{\mathrm{obs}},y_{\mathrm{mis}}) \,f(y_{\mathrm{obs}},y_{\mathrm{mis}}) \,dy_{\mathrm{mis}}.

The observed data determine the left-hand distribution, not the two components inside the integral in general. MAR places a restriction on the response probability inside the integral: it must not vary with $y_{\mathrm{mis}}$ once $y_{\mathrm{obs}}$ is fixed. But because the missing values are not recorded, the observed data alone do not show whether that restriction holds. The following example makes the issue concrete.

Worked Example: Two Different Hidden Distributions

Suppose a hypothetical population has a binary outcome $Y$, with $R=1$ when $Y$ is observed. The observed data show that 50% of cases respond, and that among respondents half have $Y=1$ and half have $Y=0$. In population-probability terms, the observed cells are $P(R=1,Y=1)=0.25$, $P(R=1,Y=0)=0.25$, and $P(R=0)=0.50$.

One possible completion assigns $Y=1$ to half of nonrespondents. Then $P(Y=1,R=0)=0.25$, and $P(Y=0,R=0)=0.25$. Overall, $P(Y=1)=P(Y=0)=0.50$, and the response probability is the same for both outcome values:

P(R=1\mid Y=1)=\frac{0.25}{0.25+0.25}=0.50, \qquad P(R=1\mid Y=0)=\frac{0.25}{0.25+0.25}=0.50.

With no covariates, this completion satisfies MCAR and therefore also MAR. Now consider a different completion: 90% of nonrespondents have $Y=1$, and 10% have $Y=0$. The observed cells do not change, but the unobserved cells become 0.45 and 0.05. The response probabilities now differ:

P(R=1\mid Y=1)=\frac{0.25}{0.25+0.45}\approx0.3571, \qquad P(R=1\mid Y=0)=\frac{0.25}{0.25+0.05}\approx0.8333.

This second completion has response depending on $Y$, so it does not satisfy MAR when there are no observed covariates. Yet both completions give exactly the same observed response proportion and the same outcome distribution among respondents. The values assigned to nonrespondents are hypothetical; the point is that the observed quantities do not distinguish these possibilities.

What Response Associations Can and Cannot Show

A response association with an observed variable can be informative. If response rates differ by $X$, that finding is inconsistent with MCAR in the population, assuming the measurements and comparisons are valid. It may be entirely compatible with MAR: response can depend on an observed predictor $X$, provided it does not additionally depend on the missing values after conditioning on the observed information.

Worked Example: Different Response Rates Across Observed Groups

In a hypothetical clinic survey, clinic type $X$ is recorded for all 200 patients, while a symptom score $Y$ is missing for some. At Clinic A, 72 of 90 patients have a recorded score; at Clinic B, 55 of 110 do. Calculate the response rates and interpret them.

For Clinic A, the rate is $72/90=0.8000$, or 80.0%. For Clinic B, it is $55/110=0.5000$, or 50.0%. The difference is $0.8000-0.5000=0.3000$, or 30.0 percentage points. Response varies across the observed clinic groups, which is evidence against MCAR if these rates reflect the target population rather than sampling fluctuation or data-processing differences.

The rates are consistent with a MAR account in which clinic type predicts response. But they do not show whether, among patients within either clinic, response also depends on their unobserved symptom scores. For patients with no score, there is no recorded $Y$ value to use in that comparison. Thus the calculation identifies an observed association; it does not verify MAR.

Likewise, a response model using observed predictors can describe and diagnose response patterns. A strong association between a recorded predictor and response may suggest that the predictor should be considered in the missing-data analysis. A weak association, or a nonsignificant test, does not establish that the missing values are irrelevant to response. Lack of detected association can also reflect limited sample size, measurement error, or an inadequate model.

When Additional Information Helps

The limitation concerns observed data alone. If missing outcomes can later be obtained for a sample of nonrespondents, those additional measurements allow direct comparisons that were previously impossible. Such follow-up can make MAR more or less plausible, but it still does not automatically prove the mechanism for every case. Its usefulness depends on how follow-up cases were selected, how accurately their outcomes were measured, and whether the followed cases represent all nonrespondents.

Worked Example: A Follow-Up Sample of Nonrespondents

A hypothetical survey records an outcome in 60 of 100 people. The 40 nonrespondents are divided evenly across two observed age groups, and a follow-up team obtains the outcome for a random sample of 5 nonrespondents from each group. In the younger group, the original respondents have a mean outcome of 12 points and the followed nonrespondents have a mean of 18 points. In the older group, the corresponding means are 20 and 21 points.

Within the younger group, the follow-up difference is $18-12=6$ points. Within the older group, it is $21-20=1$ point. These measured differences provide evidence that the distribution among at least some nonrespondents may differ from the respondent distribution within age group, especially for the younger group. If the follow-up sample was truly random within group and the outcomes were measured comparably, the differences are relevant evidence about the missing outcomes.

However, each follow-up group contains only five people, so the sample means are imprecise. Also, the conclusion relies on the stated random sampling and measurement assumptions. The follow-up data improve the evidence; they do not make the original response indicators alone a verification of MAR, nor do these small samples settle the mechanism for everyone who did not respond.

Other useful information can include external validation data, repeated attempts to contact nonrespondents, knowledge of how the data were collected, and sensitivity analyses that examine how conclusions change under alternative assumptions. Each source contributes information or makes assumptions explicit. None should be confused with a direct test of MAR using only the original observed values and response indicators.

Diagnostics, Alternatives, and Common Pitfalls

A practical assessment separates what can be checked from what must be assumed. Start with response rates by important observed groups and inspect whether response patterns differ across them. Check data collection and coding for administrative explanations, and consider whether observed variables plausibly capture predictors of both the outcome and response. These steps can reveal problems and improve a MAR-based analysis, but they do not inspect values that were never observed.

The main comparison is with MCAR. MCAR requires response to be independent of the values, observed or missing. Evidence that response varies with an observed variable can contradict MCAR; it does not contradict MAR. Conversely, failing to detect observed response associations does not establish MCAR, much less establish MAR. Little's MCAR test addresses restrictions associated with MCAR, not a general verification of MAR.

A common mistake is to argue, “The response model includes all measured predictors, and none is significant, so MAR has been confirmed.” The model can only use measured values. If the missing outcome itself influences response after conditioning on those predictors, that dependence cannot be checked by testing its coefficient, because the outcome is unavailable for nonrespondents. A more accurate conclusion is: “The observed checks did not reveal evidence against this particular observed-data specification; MAR remains an assumption.”

Another mistake is to treat a plausible mechanism as a proven one. In practice, MAR may be scientifically reasonable when response depends on variables observed for everyone and those variables capture relevant response differences. That reasoning supports a working assumption; it is not empirical verification. An analysis should state the assumption and assess how sensitive its conclusions are to departures from it, rather than reporting a data-only test as proof.

Key takeaway: Observed response patterns can expose evidence against MCAR and help assess the plausibility of MAR. But because the outcomes missing for nonrespondents are not available for comparison, observed values and response indicators alone generally cannot verify MAR.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A response rate differs by a fully observed clinic variable. What does this say about MCAR, and what does it not establish about MAR?
Why can a nonsignificant association between observed predictors and response not prove MAR?
In the binary-outcome example, why do the two proposed distributions among nonrespondents create the same observed data?
What assumptions make a follow-up sample of nonrespondents informative?
State the MAR condition for a single incomplete outcome $Y$, fully observed covariates $X$, and response indicator $R$.

Solutions. (1) A genuine response-rate difference across clinic groups is evidence against MCAR, but it does not reveal whether response also depends on missing outcomes within clinic. (2) The test uses observed variables only; it cannot check dependence on values unavailable for nonrespondents, and a nonsignificant result can also reflect limited power or model limitations. (3) Both completions keep $P(R=1,Y=1)=0.25$, $P(R=1,Y=0)=0.25$, and $P(R=0)=0.50$; only the unrecorded outcomes differ. (4) Follow-up selection should be known and suitable, outcomes should be measured comparably, and the followed nonrespondents should inform the nonrespondent group being considered. (5) MAR requires $P(R=1\mid Y,X)=P(R=1\mid X)$, with the corresponding equality for $R=0$.