Missing Data Within a Defined Estimand

The Target and the Measurement Problem

A clinical question may concern the average outcome for every person in a specified population under a specified treatment strategy. In practice, some outcomes may not be recorded. That creates a problem for estimating the target; it does not, by itself, change what the target means. Keeping these two issues separate is essential: otherwise, an analysis can quietly replace the original question with a different one about people whose outcomes happened to be observed.

Suppose \(A\) denotes assigned treatment, \(Y_i(a)\) is the outcome person \(i\) would have under treatment strategy \(a\), and \(R_i\) indicates whether that outcome is recorded, with \(R_i=1\) for recorded and \(R_i=0\) for unrecorded. Assume the outcome is defined for every person in the estimand population under each strategy being compared. For example, the endpoint might be a symptom score at a stated visit under the specified treatment strategies.

For an average-outcome treatment contrast, one possible target is

Here, the expectation is over the estimand population. The outcome scale and strategies must already be specified; the notation does not decide whether an outcome after an intercurrent event is meaningful or how that event is handled. The present issue arises once the target outcome is defined: some of its values are unavailable because they were not recorded.

In a randomized trial, assignment can support comparisons between treatment groups, but it does not guarantee that the recorded outcomes represent all assigned participants. The observed mean in arm \(a\),

is a mean among participants whose outcome was recorded in that arm. It is not automatically the target mean \(E\{Y(a)\}\). Consistency links the outcome observed under a participant’s assigned strategy to the corresponding potential outcome; it does not make missing outcomes observed.

Why Missingness Can Change an Estimate

Let \(Y\) denote the defined outcome in one treatment arm. Splitting the population according to recording status gives the identity

This follows by writing \(E(Y)=E\{YI(R=1)\}+E\{YI(R=0)\}\), where \(I(\cdot)\) equals 1 when its condition holds and 0 otherwise, and then factoring each term by the probability of its event. The observed data provide information about \(P(R=1)\) and \(E(Y\mid R=1)\). They do not directly provide \(E(Y\mid R=0)\). Unless that missing-group mean can be recovered under defensible assumptions, the full target mean is not identified by the observed mean alone.

For the observed mean to equal the target mean in a treatment arm, a sufficient condition is that recording status is independent of the outcome in that arm: \(R\perp Y\mid A=a\). Under this condition, the recorded and unrecorded groups have the same outcome mean. The condition can fail even when the recording proportion is high, and the direction of bias depends on how outcomes differ between those groups. Similar response proportions across arms do not establish this condition.

A more useful assumption may be that outcome recording is independent of the outcome after conditioning on recorded participant characteristics \(X\): \(R\perp Y\mid A,X\). This is commonly called missing at random conditional on \(A\) and \(X\). It permits the response probability to vary with observed characteristics, as long as there is no remaining dependence on the unrecorded outcome within those groups. If the outcome still predicts its own recording after conditioning on observed information, the assumption fails; this is a missing-not-at-random situation. These labels describe assumptions about the recording process, not different clinical estimands.

Recovering the Target Under an Explicit Assumption

When observed characteristics explain differences in recording, conditional adjustment can preserve the original target population. Suppose \(X\) is a baseline risk group, and \(q(x)\) is the proportion of the target population in group \(x\). If, within each treatment arm and group, the mean outcome among recorded participants equals the group’s target mean, then the arm mean can be calculated by standardizing the observed group means to the target population:

This approach requires the conditional independence assumption \(R\perp Y\mid A,X\), and positivity: each target group must have a nonzero chance of having its outcome recorded in each arm being analyzed. If a group has no recorded outcomes, its mean cannot be estimated from that group’s observed outcomes. Standardization also requires that the chosen \(X\) values and their distribution \(q(x)\) correspond to the target population.

Inverse-probability weighting is another way to use observed characteristics: recorded outcomes are weighted to represent participants with similar characteristics whose outcomes were not recorded. Like standardization, it depends on a correctly specified recording model, conditional independence of recording and outcome given the included information, and positivity. Very small estimated response probabilities can produce unstable weights and highly variable estimates. Multiple imputation is another familiar approach, but it likewise needs a model that makes the missing outcomes predictable from observed information and appropriate uncertainty calculations. None of these procedures changes the estimand; each makes assumptions to estimate it.

What Can Be Learned Without a Missingness Assumption?

When an outcome has known lower and upper limits, those limits can give bounds on the target even if the missing outcomes cannot be predicted. The bounds require that every outcome lies within the stated range, but do not require missing outcomes to resemble recorded ones. They may be wide; that width is informative because it shows how much the missing measurements leave unresolved.

Diagnostics, Comparisons, and Common Pitfalls

Useful diagnostics begin with describing who has and has not provided an outcome. Report counts and proportions by treatment arm and important observed characteristics, then inspect whether missingness is concentrated in particular groups or visits. Compare observed baseline characteristics and earlier recorded outcomes across recording status. These checks can reveal patterns that challenge a simple complete-case analysis or a proposed conditional model.

However, no comparison of observed variables can prove that outcomes are unrelated to recording after conditioning on those variables. The key diagnostic limitation is that the outcome values for \(R=0\) are unavailable, so the dependence of recording on those values cannot be checked directly. Model-based estimates should therefore be accompanied by a clear account of the assumptions and by sensitivity analyses that examine alternative plausible assumptions. If the result changes substantially across reasonable approaches, the data do not support a robust conclusion without stronger information.

The main approaches answer different practical needs:

• Complete-case analysis uses recorded outcomes only. It is simple, but targets the recorded cases directly and needs conditions for that result to represent the full target population.
• Conditional standardization or weighting uses observed characteristics to account for differing recording patterns. It can estimate the original target under conditional independence and positivity, but is vulnerable to omitted predictors, model misspecification, and sparse groups.
• Imputation fills in missing values through a statistical model. It can use observed information efficiently, but its validity depends on the imputation model and its assumptions about the missing outcomes.
• Bounds use minimal restrictions, such as a known outcome range, to show which target values remain compatible with the data. They avoid a prediction model but may be too wide to settle the clinical question.

A frequent mistake is to say, “The target mean is the mean among participants with recorded outcomes, because those are the people we can analyze.” That reasoning replaces the population in the estimand with a post-baseline subset selected by outcome recording. The correction is to state the target population and summary first, then describe the recorded subset as an incomplete source of information about that target.

A second distinction matters: an outcome that is defined but unrecorded is missing; an outcome that is not defined under the specified question is not merely a missing measurement. These require different conceptual decisions. Missing-data methods cannot, by themselves, decide what the target outcome should mean when it is undefined.