Classifying a Skipped Survey Item
A survey may record age and employment status for most respondents but leave income blank for some. The percentage with missing income describes how often the item was skipped; it does not, by itself, explain why. To classify the mechanism, identify the rule that generated each response indicator and ask whether that rule depends on the value of the item that is missing, on other observed information, or on neither.
Let \(Y_j\) denote the value of survey item \(j\), and let \(R_j=1\) mean that \(Y_j\) is observed and \(R_j=0\) mean that it is missing. Let \(X\) denote relevant information observed for the respondent, such as age group, employment status, or survey mode. For the examples below, the stated rule describes the probability of response given the relevant variables. Classification depends on that rule, not just on the realized response pattern.
In a simplified item-specific setting where \(X\) is fully observed for everyone, these definitions can be expressed in terms of the response probability. MCAR means \(P(R_j=1\mid Y_j,X)\) is constant across values of \(Y_j\) and \(X\). MAR allows this probability to vary with \(X\), but not with \(Y_j\) once \(X\) is fixed. MNAR means that, for at least some values of \(X\), the probability still varies with \(Y_j\). In a survey with several incomplete items, the full MAR condition is that the distribution of the missingness pattern, conditional on the data, depends only on values observed for that pattern. The simpler item-specific comparisons here illustrate the same logic.
Read the Response Rule, Not Just the Rate
A useful first question is: what information does the stated response rule use? If the chance of answering an item is the same for all values of the survey data, the rule is MCAR. If it varies by a fully observed predictor but not by the missing item’s value after conditioning on that predictor, the rule is MAR. If it varies with the missing value itself, even after conditioning on observed information, the rule is MNAR.
For example, for the income item, \(Y_j\) is income and \(R_j=1\) means income was reported.
Separate values observed for the respondent from the item value that may be missing.
Ask whether the chance of response still changes with the missing item’s value after that conditioning.
No dependence on survey values indicates MCAR; dependence only on observed values indicates MAR; remaining dependence on a missing value indicates MNAR.
This procedure classifies a specified response rule. In real data, the rule is rarely known in full. Associations between response indicators and observed variables can provide clues about MCAR, but they cannot generally distinguish MAR from MNAR: the value of an item when it is missing is not observed, so a dependence on that value cannot be directly checked from the observed records alone.
Worked Example: Skipped Income Depends on Observed Age Group
Hypothetical scenario. A survey records age group for every participant. For the income item, the response probability is stated to be 0.90 for participants under 50 and 0.70 for participants aged 50 or older, regardless of their actual income. Suppose 60% of participants are under 50 and 40% are 50 or older. Classify the income-item mechanism and calculate the overall response and missingness rates.
Assumptions for this classification. Age group is observed for every participant, the stated probabilities apply within age group, and the response rule does not additionally depend on actual income. These assumptions are part of the hypothetical rule; they are not conclusions that could be established from the rates alone.
Because age group is observed and the rule depends on age group but not on income after conditioning on age group, this is MAR for income. The response probability differs across observed age groups, so it is not MCAR.
The overall response probability is the expectation of the conditional response probability. With the two discrete age groups, this expectation is a weighted sum:
Thus, the overall response rate is 82%, and the missingness rate is \(1-0.82=0.18\), or 18%. This calculation describes the rate implied by the stipulated population proportions and response probabilities. It does not turn the rate into evidence that the mechanism is MAR; the classification follows from the stated dependence on observed age group and the stated lack of dependence on income.
Worked Example: Skipped Age Depends on the Age Being Asked
Hypothetical scenario. A survey asks participants to report their age. Suppose the response rule is that participants younger than 65 answer with probability 0.95, while those aged 65 or older answer with probability 0.65. Assume the age groups make up 60% and 40% of the target group, respectively. Other recorded variables are available, but the stated response rule continues to depend on the participant’s actual age within those observed values. Classify the mechanism and calculate the overall rate.
Assumptions for this classification. The age group in the rule is the participant’s true age group, not a fully observed proxy, and the stated difference in response probabilities remains after conditioning on the other observed information. The age of a person who skips the age item is therefore missing information used by the response rule.
The mechanism is MNAR for age: response probability depends on the age value that is missing for nonrespondents, even after conditioning on the other observed information. The expected overall response rate is:
The implied response rate is 83%, and the implied missingness rate is 17%. These percentages do not reveal that age-based response rule. If the difference instead depended only on a fully observed variable, such as a recorded survey mode, and not on true age after conditioning on mode, the item would be MAR rather than MNAR.
Worked Example: Health Items Missed Through an Independent Random Failure
Hypothetical scenario. A survey tablet randomly selects 10% of participant records for a brief software failure that prevents the health item from being saved. The random selection is made independently of all survey values, including health status, age, and other recorded answers. In a hypothetical group of 200 participants, 20 records are selected and the health item is missing for those records. Classify the mechanism.
Assumptions for this classification. Selection is genuinely random with respect to all survey data, and there is no additional process that makes participants with particular health values more or less likely to answer. Under this stated rule, every participant has response probability \(0.90\), regardless of the data values.
Therefore, the health item is MCAR. In this hypothetical group, the response rate is \(180/200=0.90\), or 90%, and the missingness rate is \(20/200=0.10\), or 10%. The classification does not follow merely because there are 20 missing values. It follows because the mechanism that selected the records is stipulated to be independent of the survey data. In an actual survey, a software failure tied to a particular clinic or time could be associated with observed characteristics, so the independence assumption would need separate consideration.
Why the Missingness Rate Cannot Classify the Mechanism
The general identity for an overall response rate is the expectation of the conditional response probability:
This form applies whether \(Y_j\) and \(X\) are continuous, discrete, or a mixture. If both are discrete, the expectation can be written as a weighted sum over their possible values; for continuous variables, it corresponds to averaging over their joint distribution. The overall rate combines response probabilities across people with different values. It does not show which values produced higher or lower probabilities.
For example, suppose half of a hypothetical population has low income and half has high income. A rule that gives response probabilities 0.90 for low income and 0.70 for high income has an overall response probability of:
A different rule that gives every person response probability 0.80 also has an overall response probability of 0.80. The first rule depends on income itself and is MNAR for the income item if that dependence remains after conditioning on observed information; the second rule is MCAR under the stated independence assumption. A third rule could have the same overall rate while varying response probability only by a fully observed predictor, making it MAR. Thus, identical missingness rates can arise under different mechanisms.
The relevant comparison is not “what fraction is missing?” but “how does the response probability vary, and with respect to which values?” A small missingness rate does not make an MNAR mechanism MAR, and a large missingness rate does not by itself imply MNAR.
Diagnostics, Comparisons, and Limits
A practical review begins by tabulating response indicators against variables observed for respondents and nonrespondents. For example, one can compare income-item response rates by observed age group or health-item response rates by recorded survey mode. Clear differences can contradict MCAR, because MCAR requires response to be independent of the survey values. A lack of detected differences does not prove MCAR: the sample may be small, the relevant variable may not have been measured, or the pattern may be subtle.
Observed-data diagnostics do not establish MAR. A response association with observed age is consistent with MAR, but it could also coexist with an additional dependence on missing income. Conversely, a response indicator that shows little association with recorded variables could still depend on the missing item itself. This is the central limit of classifying mechanisms from observed survey records.
| Mechanism | What the response rule depends on | Survey-item illustration |
|---|---|---|
| MCAR | No survey data values, observed or missing | A genuinely data-independent random failure prevents saving the health item |
| MAR | Observed values, but not missing values after conditioning on observed data | Income response probability varies by fully observed age group, not by income within age group |
| MNAR | At least one value that is missing, even after conditioning on observed data | Age response probability varies by true age after accounting for other observed information |
These labels help state assumptions about the response process; they do not, by themselves, determine whether a particular analysis is appropriate or how sensitive a conclusion is to those assumptions. For an item with a plausible MNAR rule, the unobserved values and response process cannot generally be recovered from the observed records alone. Any analysis that relies on a more restrictive mechanism should therefore make that assumption explicit and consider how conclusions might change under alternatives.
A common mistake is to call a variable MAR because the response rate differs across groups, or to call a variable MCAR because the overall missingness rate is low. Both arguments use the size or observed pattern of missingness as if it specified the response rule. The correction is to identify the rule’s inputs: differences explained only by observed data are compatible with MAR; dependence on the item’s missing value indicates MNAR; independence from all survey values is required for MCAR.
Check Your Understanding
For each scenario, classify the item mechanism from the stipulated rule. Then compare your reasoning with the solutions below.
- A survey’s income item has a response probability of 0.80 for every participant, independent of all survey values. What is the mechanism?
- For a health item, participants with a fully observed indicator of online versus paper completion have different response probabilities. Within each completion mode, response probability does not depend on health status. What is the mechanism?
- For an income item, response probability is lower for people with higher true income, even after conditioning on all recorded variables. What is the mechanism?
- Two surveys each have a 15% missingness rate for age. Does that rate establish that their age mechanisms are the same? Explain.
Solutions. (1) MCAR: the response probability is constant across all survey values by stipulation. (2) MAR: completion mode is observed, and the rule has no remaining dependence on the missing health value after conditioning on mode. (3) MNAR: response probability still depends on income, the item value that may be missing, after conditioning on observed variables. (4) No. The same marginal rate can result from different response rules, including MCAR, MAR, or MNAR; the rate alone does not identify the mechanism.