Why Compare Response Across Groups?
Suppose a survey asks every participant for an outcome, such as annual income, and also records each participant's age group. If the outcome response rate differs between age groups, the pattern is relevant to missingness: it challenges the idea that outcome response is completely unrelated to the data. But it does not, by itself, show that response depends on the unreported outcome. The distinction is between MCAR, which allows no dependence on any data values, and MAR, which allows dependence on values that are observed.
The comparison is meaningful only if group membership is known for respondents and nonrespondents alike. Otherwise, the denominator for each group's response rate is not available from the records. In what follows, consider one outcome \(Y\) that may be missing and a group variable \(X\) observed for every eligible person. \(X\) may represent age group, treatment arm, clinic, or another recorded characteristic.
For a sample, calculate the group response rate as the number with observed \(Y\) in the group divided by the total number of eligible people in the group. The denominator includes both respondents and nonrespondents. A percentage calculated using respondents alone cannot describe the response rate.
What MCAR and MAR Imply About Group Rates
For this one-outcome setting, MCAR means that response is independent of the complete data, including \(Y\) and \(X\). MAR means that, conditional on the observed information \(X\), response does not additionally depend on \(Y\). These definitions concern the population missingness mechanism, not just the pattern in one sample.
The implication for group response rates follows directly. Under MCAR, average over the possible values of \(Y\) within a group:
For a continuous \(Y\), the sum is replaced by an integral; the same argument applies. Consequently, MCAR implies the same population response probability in every group defined by \(X\). Under MAR, however, \(P(R=1\mid X=x)\) can change with \(x\). If age group predicts response, for example, response can be MAR when that group is observed and accounts for the response differences. The missing outcome itself must not provide additional information about response after conditioning on the observed group.
This is an implication, not a way to certify a mechanism. Equal group response rates do not prove MCAR: response could differ by \(Y\) within groups, or by some other feature of the complete data. Unequal rates in the population contradict MCAR for these groups, but can be compatible with MAR. Sample rates need additional care: unequal percentages in a finite sample can arise even if the population response probability is identical.
Reading Group Response Rates
Start with a clear table: for every group, give the number of eligible people, the number who responded, and the rate calculated from those counts. Check that the groups are mutually exclusive and that all eligible people appear in exactly one group. Then describe what the observed pattern can and cannot establish. The calculations summarize response in the sample; they do not reveal the missing \(Y\) values.
Worked Example: Unequal Rates as Evidence Against MCAR
In a hypothetical survey, age group is recorded for all 100 eligible participants, but annual income is missing for some. In the younger group, 45 of 50 participants report income. In the older group, 30 of 50 report income. Calculate and compare the response rates.
Younger group: \(45/50=0.900\), or \(90.0\%\).
Older group: \(30/50=0.600\), or \(60.0\%\).
The observed difference is \(0.900-0.600=0.300\), or 30.0 percentage points. If these rates represented the population probabilities, they would be unequal, contradicting MCAR because age group is part of the complete data. The pattern could be MAR if age group explains the response difference and, within age group, response does not additionally depend on income. The sample percentages alone do not show whether that conditional independence holds or whether the population rates truly differ.
Worked Example: A Group Difference Consistent With MAR
Consider a hypothetical population divided into groups A and B. In each group, half have a low value of \(Y\) and half a high value. Suppose the response probabilities are 0.90 in group A and 0.60 in group B, regardless of whether \(Y\) is low or high. Show how the pattern can satisfy MAR while not satisfying MCAR.
In group A, \(P(R=1\mid Y=\text{low},X=A)=0.90\) and \(P(R=1\mid Y=\text{high},X=A)=0.90\). In group B, both corresponding probabilities equal 0.60. Thus, for either group, changing \(Y\) while holding \(X\) fixed does not change the response probability: \(P(R=1\mid Y,X)=P(R=1\mid X)\), which is the MAR condition.
The marginal group response probabilities are nevertheless different: \(P(R=1\mid X=A)=0.90\), whereas \(P(R=1\mid X=B)=0.60\). Since MCAR requires response probability not to vary with \(X\), this mechanism is not MCAR. The example shows why unequal group rates can be consistent with MAR: the observed group predicts response, but the missing outcome does not predict response after accounting for group.
Why Group Rates Cannot Establish MAR
A table of response rates gives one average for each group. MAR is a stronger claim: response must be independent of the missing outcome conditional on the observed information. Since \(Y\) is unobserved when \(R=0\), response rates by \(X\) cannot directly compare respondents' and nonrespondents' \(Y\) values. Different assumptions about those missing values can therefore fit the same observed response-rate table.
Worked Example: The Same Rates Can Hide Outcome Dependence
In a hypothetical group, 80 of 100 people respond, so the response rate is 80%. Among respondents, 40 report a high value of binary outcome \(Y\), and 40 report a low value. The outcomes of the 20 nonrespondents are unavailable. Compare two possible completions of the missing outcomes.
Completion 1: Suppose 10 nonrespondents would have had high \(Y\), and 10 low \(Y\). Then the full group has 50 high and 50 low values. Among the high values, 40 of 50 are observed; among the low values, 40 of 50 are observed. Both response probabilities are \(40/50=0.800\). In this completed group, response is independent of \(Y\), consistent with MAR when conditioning on this group.
Completion 2: Suppose instead that all 20 nonrespondents would have had high \(Y\). The full group then has 60 high and 40 low values. Among high values, \(40/60=0.667\) are observed; among low values, \(40/40=1.000\) are observed. Response now depends on \(Y\) within the group, which is inconsistent with MAR conditional on group.
Both completions produce exactly the observed record: 80 respondents, 40 observed high outcomes, and 40 observed low outcomes. The response rate is 80% in either case. The example does not say which completion is true; it shows why observed group rates alone cannot establish whether MAR holds.
Assumptions, Checks, and Common Pitfalls
A group-rate comparison is interpretable when the group variable is genuinely observed for respondents and nonrespondents, the target population and eligibility rules are consistent across groups, and the denominator includes everyone at risk of responding. If \(X\) is missing for some people, a comparison restricted to known \(X\) may not represent all eligible participants. Likewise, a group variable recorded only after response cannot straightforwardly serve as observed information explaining response for nonrespondents.
Treat observed differences as evidence, not a deterministic verdict about the population. Even under MCAR, sample response rates will usually not be exactly equal: random samples fluctuate. A substantial or systematic difference may make MCAR less plausible, but judging its strength requires considering sample sizes and sampling uncertainty. Conversely, rates that look similar can conceal meaningful differences when groups are small, and equal rates do not rule out response depending on \(Y\) within groups.
The relevant group definition also matters. If response varies by a recorded variable omitted from the comparison, rates aggregated over that variable can obscure differences. And if a group contains different subgroups with different response patterns, its overall rate is a weighted average of their rates; it does not establish independence within the group. The MAR conditioning set must include the observed information relevant to response, not merely one convenient grouping variable.
| Observed pattern | What it implies | What it does not imply |
|---|---|---|
| Unequal population response probabilities across observed groups | MCAR is incompatible with the group differences. | It does not prove MAR or identify why response varies. |
| Unequal sample response rates across groups | The observed data show a group difference worth assessing. | It does not alone prove unequal population rates or rule out MCAR. |
| Similar response rates across groups | No difference is apparent in this comparison. | It does not prove MCAR or MAR. |
A common mistake is to reason, “Response rates differ by age, so the missingness is MAR.” The correction is that the difference rules out MCAR only if it reflects unequal population probabilities; it is compatible with MAR, but MAR additionally requires no dependence on the missing outcome after conditioning on all relevant observed information. Another mistake is to calculate the rate among respondents in each group. The correct denominator is all eligible members of that group, including those with missing outcomes.
This comparison is one diagnostic, not a substitute for specifying the missingness mechanism. MCAR, MAR, and MNAR describe different dependence structures: a group difference can be explained by observed \(X\), but response might also depend on unreported \(Y\). Since the latter values are missing, the rate table cannot settle that question. Group rates are most useful as a transparent check on whether a simple MCAR assumption is plausible and as a prompt to consider observed predictors of response.
Check Your Understanding
Answer each question using the distinction between a sample pattern and a population mechanism.
- A fully observed clinic indicator has population response probabilities of 0.75 in Clinic A and 0.50 in Clinic B. Is MCAR compatible with these probabilities? Could MAR be compatible?
- In a sample, response rates are 82% and 79% in two groups. Does this prove MCAR? Explain.
- A response-rate table uses the number of respondents in each group as its denominator. What is wrong, and what denominator should be used?
- Why can equal response rates across observed groups fail to establish MAR?
Solutions. 1. MCAR is incompatible with the unequal population probabilities because MCAR requires response probability to be constant across groups. MAR could be compatible if response is independent of the missing outcome conditional on clinic. 2. No. The rates are sample estimates and may differ through sampling variation; moreover, similar rates do not rule out dependence on the missing outcome. 3. That denominator excludes nonrespondents and therefore cannot give the proportion of eligible people who responded. Use all eligible members of the group, respondents and nonrespondents. 4. Equal rates only compare average response across those groups. Response could still depend on the missing outcome within groups, or on another feature of the data not included in the comparison.