Diagnosing Missingness Patterns in Causal Data

Why Diagnose Missingness?

In a causal study, missing values can occur in treatment, covariates, outcomes, or several of these at once. A useful first step is to describe which variables are missing, for whom, and when. Those descriptions can reveal where incompleteness is concentrated and help identify which observed information predicts it. They do not, by themselves, reveal the unobserved values or prove the process that generated the missingness.

Let \(V\) denote the full data that would be available if every study variable were measured. For variable \(V_j\), define \(R_j=1\) when its value is observed and \(R_j=0\) when it is missing. A missingness pattern is the vector of these indicators for one participant. In longitudinal data, the indicators can also be indexed by visit.

These definitions concern the full data and the process that determines what is recorded. Diagnostics use the observed data. They can show that missingness is associated with recorded treatment, covariates, or prior measurements; they cannot directly compare missingness with values that were never observed.

Start with Rates and Joint Patterns

For each study variable, report the number and proportion observed and missing. State the denominator: it is usually all eligible participants, but it may differ if eligibility or follow-up begins at different times. Also summarize joint patterns, such as “outcome missing, covariates observed.” Separate variable-specific rates matter because one overall percentage can conceal that an outcome is often missing while treatment is nearly always recorded.

Next, compare rates across scientifically relevant observed groups: for example, treatment arm, site, baseline severity, or visit. Give group counts as well as percentages. A percentage based on a small group can be unstable, and a difference in percentages is a description, not proof that the grouping variable caused missingness.

Modeling Whether a Variable Is Observed

A regression model can summarize how observed variables predict an observation indicator. For a binary indicator \(R_j\), a common descriptive model is logistic regression. With fully observed predictors \(Z\), it has the form

Here the modeled event is explicitly “the variable is observed.” A positive coefficient corresponds to higher conditional odds of observation as that predictor increases, holding the other included predictors fixed. If the event is instead defined as missingness, the coefficient direction reverses. Report which event was modeled, the predictors, and preferably predicted observation rates or odds ratios in addition to coefficients.

Before interpreting such a model, check that its predictors are available for the records used in the fit, the outcome indicator is coded consistently, and the functional form is suitable for the intended description. A standard logistic model also assumes independent participant-level observations; with clustered or repeated observations, the dependence needs to be handled in the model or uncertainty estimates. Poor fit, sparse groups, or omitted predictors can make the summary misleading. Even a well-fitting model remains an association model for observation, not a causal model of why values are missing.

Use Time to Describe Longitudinal Patterns

For repeated measurements, a single overall missingness rate discards the order in which values are missing. Report observation at each scheduled visit, using a clear denominator, and summarize each participant’s sequence of observed and missing measurements. For example, \(1110\) means observed at the first three visits and missing at the fourth.

A pattern is monotone when, after the first missing visit, all later scheduled measurements are missing. A pattern is intermittent when a measurement is missing and a later one is observed. These labels describe the sequence; neither label identifies the reason for missingness. Be precise about denominators: a visit-specific rate among all enrolled participants differs from the rate among those still under follow-up.

Why Diagnostics Cannot Establish MAR

Observed-variable prediction is informative because it can reveal that missingness is not independent of recorded information. If, for example, observation rates differ across fully observed severity groups, that association is incompatible with MCAR for those data under the usual definition. But the finding does not discriminate MAR from MNAR: both mechanisms can produce the same observed association.

The limitation can be seen with a binary outcome \(Y\) that is sometimes missing and its indicator \(R\). The observed data reveal \(P(R=1)\) and \(P(Y=1\mid R=1)\). They do not reveal \(P(Y=1\mid R=0)\), because \(Y\) is unobserved when \(R=0\). For instance, suppose \(P(R=1)=0.8\) and \(P(Y=1\mid R=1)=0.5\). The observed data are unchanged whether the missing group has \(P(Y=1\mid R=0)=0.1\), \(0.5\), or \(0.9\). Yet these choices imply different relationships between \(Y\) and \(R\), and thus different possible missingness mechanisms.

Therefore, a test based only on observed data cannot generally establish MAR. An omnibus test designed to assess MCAR tests an MCAR null under its own distributional assumptions; rejecting it can be evidence against MCAR, while failing to reject is not proof of MCAR and does not demonstrate MAR. The same caution applies to a nonsignificant coefficient in a missingness regression: it can reflect limited precision, model misspecification, or unmeasured predictors, rather than independence from the full data.

This is why diagnostics should be reported as descriptions of observed patterns, with their assumptions and limitations. They can identify where missingness is concentrated and whether measured variables predict observation. A conclusion about MAR requires assumptions that cannot be verified from the observed data alone, or additional information such as follow-up or validation measurements.

Common Mistakes

• Calling predictable missingness proof of MAR. A predictor associated with observation is evidence against MCAR in that respect, but missing values may still influence observation.
• Treating a nonsignificant test as proof of no association. A test’s failure to reject can reflect low power or an inadequate model; it does not certify MCAR or MAR.
• Reporting only one overall percentage. It can obscure which variables and which joint patterns are incomplete. Give variable-specific rates and relevant cross-tabulations.
• Ignoring time order. A person who misses one visit and returns differs descriptively from someone with permanent dropout. Summarize sequences, not only visit totals.
• Giving a causal interpretation to a missingness model. A regression of observation on recorded variables is descriptive unless a separate causal design and assumptions justify more.