Missingness Mechanisms for Repeated Outcomes

Why the Missingness Mechanism Matters

A panel can be unbalanced for many reasons: participants may leave a study, miss one interview, or stop reporting an outcome after a particular event. The same observed pattern of gaps can arise from very different processes. For example, a missed interview might be unrelated to a participant’s outcomes, related to outcomes recorded at earlier interviews, or related to the outcome that would have been recorded at the missed interview. Those distinctions affect what information an analysis may use to account for missing observations.

The key is to describe missingness relative to the information available in the complete data and in the observed data. The terms MCAR, MAR, and MNAR name three such relationships. They are assumptions about how observation patterns arise, not labels that can usually be verified just by looking at the gaps.

Panel Notation and Observation Patterns

Consider \(N\) panel units observed over periods \(t=1,\ldots,T\). Let \(Y_{it}\) be the outcome for unit \(i\) at time \(t\), and let \(X_{it}\) denote covariates. Define \(R_{it}=1\) if \(Y_{it}\) is observed and \(R_{it}=0\) if it is missing. The vector \(R_i=(R_{i1},\ldots,R_{iT})\) records the unit’s observation pattern. For example, \(R_i=(1,1,0,0)\) means that the first two outcomes are observed and the last two are missing.

Write \(Y_i=(Y_{i1},\ldots,Y_{iT})\) for the complete outcome vector. For a particular pattern \(r\), let \(Y_{i,\mathrm{obs}(r)}\) denote the outcomes observed under that pattern and \(Y_{i,\mathrm{mis}(r)}\) the outcomes missing under it. If all covariates are observed, they may be included as \(X_i\) in the information used to describe the mechanism. More generally, any covariate that is itself missing must be treated according to whether its value is available for the relevant pattern.

Definition: A missingness mechanism is the conditional distribution of the observation pattern \(R_i\) given the complete data on the unit. It describes how the probability of each pattern varies with outcomes and covariates. In this tutorial, “observed data” means the outcomes and covariates actually available under the realized pattern.

MCAR: Missing Completely at Random

Missing completely at random means that the observation pattern is independent of the complete outcomes and covariates. Formally, for every possible pattern \(r\),

P(R_i=r\mid Y_i,X_i)=P(R_i=r).

Thus, among units in the panel, the probability of a particular observation pattern does not vary with either their observed or unobserved outcomes or their covariates. A random administrative failure that removes records without regard to the units or their data is a possible MCAR mechanism. In practice, such a strong independence assumption is often difficult to justify.

Worked Example: Checking an MCAR Description

Suppose a hypothetical four-period panel has only four possible observation patterns: \(1111\), \(1100\), \(1000\), and \(0000\). The study’s record system assigns these patterns with probabilities \(0.70\), \(0.15\), \(0.10\), and \(0.05\), respectively, independently of each unit’s outcome path and covariates. Is the mechanism MCAR?

The probabilities sum to one: \(0.70+0.15+0.10+0.05=1.00\). By the stated design, for any complete outcome path \(y_i\) and covariate values \(x_i\),

P(R_i=1100\mid Y_i=y_i,X_i=x_i)=0.15, \qquad P(R_i=0000\mid Y_i=y_i,X_i=x_i)=0.05.

The corresponding probabilities for \(1111\) and \(1000\) are likewise \(0.70\) and \(0.10\), regardless of \(y_i\) and \(x_i\). The conditional pattern probabilities therefore equal their unconditional probabilities. This mechanism is MCAR. The example specifies an assumption about how patterns are generated; seeing those four patterns in a data set, by itself, would not establish that assumption.

MAR: Missing at Random Given Observed Information

Missing at random means that, once the observed outcomes and available covariates are held fixed, the pattern probability does not depend further on the outcomes that are missing under that pattern. When covariates are fully observed, the pattern-specific condition is

P(R_i=r\mid Y_i,X_i) = P(R_i=r\mid Y_{i,\mathrm{obs}(r)},X_i).

The right-hand side may vary across observed outcome histories and covariates. MAR does not mean that missingness is unrelated to outcomes. It permits, for example, a later outcome to be more likely missing for units whose earlier recorded outcome was high. What it rules out is additional dependence on the unobserved outcomes after conditioning on the information shown on the right-hand side.

In a sequential dropout setting, the observation decision at time \(t\) can be described using information available before that decision, such as baseline characteristics and previously observed outcomes. If a current outcome is observed before a later interview decision, it may be part of the history available for that later decision. The precise history therefore depends on the timing of measurement and dropout; it should not automatically include data that would not yet have been observed.

Worked Example: Dropout Depending on Observed History

Consider a hypothetical two-period panel in which \(Y_{i1}\) is observed for every unit and \(Y_{i2}\) may be missing. Let \(X_i\) be a fully observed baseline covariate coded 0 or 1. Suppose the probability that \(Y_{i2}\) is missing is given by this table:

Observed first-period outcome	\(X_i=0\)	\(X_i=1\)
Low	0.10	0.20
High	0.20	0.30

Assume that these probabilities do not depend on \(Y_{i2}\) once \(Y_{i1}\) and \(X_i\) are fixed. For a unit with a high first-period outcome and \(X_i=1\),

P(R_{i2}=0\mid Y_{i1}=\text{high},X_i=1,Y_{i2}) =0.30

for every possible value of \(Y_{i2}\). The probability of observing the second outcome is \(1-0.30=0.70\). This same pattern probability applies whether the unobserved second-period outcome would have been low or high. Because the missingness probability depends on observed history and a fully observed covariate, but not further on the missing outcome, this is MAR. It is not MCAR: the missingness probability changes across the table as \(Y_{i1}\) and \(X_i\) change.

MNAR: Dependence on Unobserved Outcomes

Missing not at random means that missingness still depends on an outcome that is unobserved, even after conditioning on the observed outcomes and covariates. In the pattern-specific notation, the missingness probability cannot be expressed using only the observed information:

P(R_i=r\mid Y_i,X_i) \ne P(R_i=r\mid Y_{i,\mathrm{obs}(r)},X_i)

for at least some values of the complete data. For example, participants with unusually high current symptoms might be less likely to complete that same interview, even after accounting for their previous recorded symptoms and baseline covariates. The missing current symptom value is then relevant to the probability that it is missing. MNAR is also called nonignorable missingness in many settings.

Worked Example: Dependence on the Missing Outcome

Suppose in a hypothetical two-period panel that \(Y_{i1}\) and \(X_i\) are observed for everyone, but \(Y_{i2}\) may be missing. Consider two possible categories for \(Y_{i2}\), low and high. Holding the observed history \((Y_{i1},X_i)\) fixed, suppose the probability of missing the second outcome is \(0.10\) when \(Y_{i2}\) is low and \(0.50\) when \(Y_{i2}\) is high.

For the two possible complete outcomes, the conditional probabilities are

P(R_{i2}=0\mid Y_{i1},X_i,Y_{i2}=\text{low})=0.10, \qquad P(R_{i2}=0\mid Y_{i1},X_i,Y_{i2}=\text{high})=0.50.

The observed history is identical in these comparisons, but the missingness probability differs by \(0.50-0.10=0.40\). The difference depends on \(Y_{i2}\), which is unavailable precisely when \(R_{i2}=0\). Conditioning on the observed history and covariates therefore does not remove the dependence on the missing outcome: this is MNAR. The probabilities are a hypothetical specification of the mechanism, not quantities that can be read directly from the missing outcomes in an observed data set.

How the Three Mechanisms Relate

MCAR, MAR, and MNAR distinguish which information may explain the observation pattern. In particular, MCAR is a special case of MAR: a pattern probability that is constant across complete data is also constant when the observed outcomes and covariates are held fixed.

\begin{aligned} \text{MCAR:}\quad P(R_i=r\mid Y_i,X_i)&=P(R_i=r),\\ \text{MAR:}\quad P(R_i=r\mid Y_i,X_i)&=P(R_i=r\mid Y_{i,\mathrm{obs}(r)},X_i). \end{aligned}

To see the implication, under MCAR the left side is already a constant with respect to \(Y_i\) and \(X_i\). Conditioning that same constant on observed outcomes and covariates leaves it unchanged. Thus it satisfies the MAR equality as well. MAR allows more dependence than MCAR: it allows the observation pattern to vary with information that is observed. MNAR allows residual dependence on the unobserved outcomes.

Mechanism	Information that may affect the pattern probability	Dependence on missing outcomes after conditioning on observed information
MCAR	Neither outcomes nor covariates	No
MAR	Observed outcomes and available covariates	No
MNAR	Observed information and, additionally, unobserved outcomes	Yes

This comparison concerns the missingness mechanism, not whether a particular estimator is unbiased or consistent. A mechanism label alone does not specify the outcome model, the target estimand, or the other assumptions needed for a particular analysis.

What Can Be Conditioned On?

The practical distinction is which information can be used to account for variation in observation patterns. Under MCAR, the mechanism itself provides no outcome or covariate predictors of missingness. Under MAR, one may condition on observed outcome history and covariates that are available under the observation pattern. That can include baseline covariates and previously observed time-varying covariates; it can also include an outcome from an earlier period if it was recorded before the later missingness decision.

Under MNAR, conditioning on observed history and covariates is not sufficient to describe the mechanism. Its dependence on the missing outcome must be represented through assumptions involving that outcome or a model for the selection process. The missing outcome cannot simply be included as if it were observed. More detailed estimation strategies are separate questions; identifying the mechanism clarifies what assumptions those strategies would need.

What the Observed Data Can and Cannot Establish

Observed records reveal which outcomes are missing and the values of observed outcomes and covariates. They do not reveal the values of outcomes in the missing cells. Consequently, MAR and MNAR are generally not distinguishable from observed data alone without additional assumptions, design information, or external information. A pattern in which dropout is associated with earlier observed outcomes is consistent with MAR, but that association does not rule out an additional dependence on the missing outcomes.

Similarly, a failure to find an association between missingness and observed covariates does not prove MCAR. The mechanism could depend on unobserved outcomes, or an association might be difficult to detect in a finite sample. Mechanism claims should therefore be justified using knowledge of how data collection and nonresponse occur, not just a test on recorded variables.

Common Mistake: Treating Observed Predictors as Proof of MAR

A common but incorrect argument is: “Dropout depends on baseline risk, so the mechanism is MAR.” The fact that dropout depends on an observed baseline covariate is not enough. If dropout also depends on the outcome that would have been measured but was not, even after conditioning on baseline risk and observed history, the mechanism is MNAR. The correction is to ask whether the missing outcome still predicts its own missingness after all relevant observed information has been held fixed.

Key takeaway: MCAR allows no dependence on the complete data; MAR allows dependence on observed outcomes and available covariates but none on missing outcomes after conditioning on them; MNAR retains dependence on unobserved outcomes. These are substantive assumptions about the observation process, not properties established simply by the shape of an unbalanced panel.

Check Your Understanding

For each question, distinguish the information available at the missingness decision from the outcomes that remain unobserved.

In a three-period panel, explain what the pattern \(R_i=(1,0,1)\) says about which outcomes are observed.
A missed third-period interview depends on the first-period outcome, which is observed, but not on the third-period outcome after conditioning on the first. Is this MCAR, MAR, or MNAR? Explain.
Why is MCAR a special case of MAR?
Dropout varies with an observed baseline covariate. What additional question must be answered before calling the mechanism MAR?
Why can the observed data generally not establish that a mechanism is MAR rather than MNAR?