Causal Estimands With Incomplete Measurements

Introduction

A causal question concerns what would happen under specified interventions in a specified population. Missing values concern what was recorded about that population. These are related problems, but they are not the same one: the absence of a recorded outcome does not, by itself, change which outcome contrast is the scientific target.

Keeping the target separate from the observed data is essential. A difference calculated among people whose treatment and outcome are recorded may be a useful summary of those records, but it need not equal the causal contrast for the population of interest. The same distinction applies when covariates are missing and the observed covariate distribution differs from the target population’s distribution.

Define the Target Before Looking at What Is Recorded

Let \(A\) denote a binary treatment, with values 1 and 0, and let \(Y\) be an outcome measured at a specified time. For each member of the target population, \(Y(1)\) is the outcome that would occur under treatment 1, and \(Y(0)\) is the outcome that would occur under treatment 0. Only one of these outcomes is observed for a person, but both potential outcomes define the causal contrast.

Definition: The average treatment effect (ATE) in a target population is the difference between the population mean outcome under treatment 1 and the population mean outcome under treatment 0:

\operatorname{ATE}=E^*[Y(1)-Y(0)] =E^*[Y(1)]-E^*[Y(0)].

Here \(E^*\) denotes expectation over the specified target population. The outcome scale, treatment contrast, outcome time, and target population are part of the definition of the estimand. If \(Y\) is measured in points, for example, the ATE is measured in points.

The ATE is a target estimand: a population quantity defined by the causal question. It is not an observed-data formula or an estimation procedure. Whether the available data can identify or estimate it is a separate issue. In particular, the existence of missing values does not redefine the potential outcomes or automatically replace the target population with the people whose values happen to be recorded.

To represent incomplete records, let \(R_A=1\) mean that treatment is recorded, \(R_Y=1\) mean that the outcome is recorded, and \(R_X=1\) mean that the covariate value is recorded. A zero indicates that the corresponding value is unavailable. These indicators describe data availability; they are not treatment assignments or outcomes. For instance, \(R_Y=0\) does not mean \(Y=0\).

Observed-Data Quantities Are Different Objects

An observed-data quantity is calculated from the values that are actually available. If outcomes are recorded only for some people, the observed outcome mean is \(E[Y\mid R_Y=1]\), not automatically the target-population mean \(E^*[Y]\). If treatment is missing, a treated-versus-untreated comparison among people with recorded treatment uses a selected set of records. If a covariate is missing, a calculation based on recorded covariates may use different population weights from the target estimand.

A simple identity shows why even an ordinary mean can change when it is calculated only among responders. Suppose \(Y\) is defined for the target population, \(R_Y\) is binary, and \(P(R_Y=1)>0\). Then:

\begin{aligned} E[Y\mid R_Y=1]-E[Y] &=\frac{E[YR_Y]}{P(R_Y=1)}-E[Y]\\ &=\frac{E[YR_Y]-P(R_Y=1)E[Y]}{P(R_Y=1)}\\ &=\frac{\operatorname{Cov}(Y,R_Y)}{P(R_Y=1)}. \end{aligned}

The first equality follows because \(YR_Y=Y\) when \(R_Y=1\) and is zero otherwise. The final expression follows from the definition of covariance. Thus the responder mean equals the target mean when this covariance is zero, but it need not do so. This identity concerns a mean, not a causal effect; for a causal contrast, both treatment groups and the relevant potential outcomes must be considered.

Worked Example: A Missing Outcome Changes the Observed Contrast

Consider a hypothetical finite population of eight people. Four have potential outcomes \(Y(0)=0\) and \(Y(1)=2\); the other four have \(Y(0)=10\) and \(Y(1)=12\). Treatment is assigned to four people, with two from each group; the other four receive control, also with two from each group. The outcome is recorded for the two high-outcome people assigned treatment and the two low-outcome people assigned control. All other outcomes are missing.

Target estimand. The mean potential outcome under control is \(E^*[Y(0)]=(4\cdot 0+4\cdot 10)/8=5\). Under treatment it is \(E^*[Y(1)]=(4\cdot 2+4\cdot 12)/8=7\). Therefore:

\operatorname{ATE}=7-5=2.

The target ATE is 2 outcome units. This calculation uses the full finite population’s potential outcomes to specify the target; it does not claim that both potential outcomes are observed for each person.

Observed-data contrast. The two recorded treated outcomes are both 12, so their mean is 12. The two recorded control outcomes are both 0, so their mean is 0. The difference in recorded group means is \(12-0=12\) outcome units. It is not the target ATE of 2. The example illustrates that an observed contrast can differ sharply from the causal estimand when outcome recording selects different people in the treatment groups.

Missing Treatment and Missing Covariates

When treatment is missing, the causal question can still be stated in terms of interventions \(A=1\) and \(A=0\), provided those interventions are well-defined. The missing treatment value makes it unclear which observed treatment group a person belongs to; it does not itself alter the intervention contrast. An analysis using only known treatment values therefore refers directly to the records with \(R_A=1\), unless additional assumptions justify a link to the target population.

When covariates are part of the target definition or the adjustment strategy, their distribution also matters. For example, write \(X\) for a baseline covariate and define the conditional causal effect at \(X=x\) as \(\tau(x)=E[Y(1)-Y(0)\mid X=x]\). The target ATE can be expressed by averaging these conditional effects over the target covariate distribution:

\operatorname{ATE}=E^*[\tau(X)] =\sum_x \tau(x)P^*(X=x)

for a discrete covariate. This is an identity that averages stratum-specific effects using target-population weights. When the covariate is observed for only a subset, the distribution of \(X\) among people with recorded covariates, \(P(X=x\mid R_X=1)\), need not equal \(P^*(X=x)\). Substituting the former weights changes the quantity being calculated. The formula does not, by itself, say whether the target ATE is identifiable from the available records; that requires assumptions connecting the observed data to the target population.

Worked Example: Missing Treatment Values

In a hypothetical study, outcome values are available for 100 people, but treatment is recorded for only 80. Among the 40 people with recorded treatment \(A=1\), the mean outcome is 9 units. Among the 40 with recorded \(A=0\), it is 6 units. The 20 people with missing treatment have mean outcome 7 units.

The contrast among people with recorded treatment is:

9-6=3\text{ outcome units}.

This is a difference in observed outcome means for the 80 people whose treatment is known. It is not, from these figures alone, the target-population ATE: the treatment statuses of 20 people are unknown, and the group means do not provide both potential outcomes for anyone. For example, assigning all 20 unknown treatment values to the control group would produce a control mean of \((40\cdot 6+20\cdot 7)/60=6.333\) units, and a known-or-assigned-group contrast of \(9-6.333=2.667\) units. That different number comes from a different classification rule, not from new information about the causal effect.

Worked Example: Covariate Weights and the Target ATE

Suppose a hypothetical target population is evenly split between two baseline covariate strata, \(X=L\) and \(X=H\). Assume the stratum-specific causal effects are 0 outcome units for \(L\) and 4 outcome units for \(H\). These effects are defined for the target population; the example focuses only on how the averaging weights affect the estimand.

The target-population ATE weights each stratum by one half:

\operatorname{ATE} =(0.5)(0)+(0.5)(4)=2\text{ outcome units}.

Now suppose the covariate is recorded for a subset in which 20% are in stratum \(L\) and 80% are in stratum \(H\). If the stratum-specific contrasts are averaged using this recorded-subset distribution, the resulting observed-data standardized contrast is:

(0.2)(0)+(0.8)(4)=3.2\text{ outcome units}.

The value 3.2 uses the recorded subset’s covariate composition; the target ATE of 2 uses the target population’s composition. The discrepancy arises from the weights, even though the stratum-specific effects in this example are the same in both calculations. The recorded-subset calculation is not automatically the target estimand.

Interpretation and a Common Pitfall

An estimand should be described in terms of the population and intervention contrast it represents. An observed-data quantity should be described in terms of the records used to calculate it. Keeping those descriptions separate avoids claiming that a complete-record mean, a known-treatment contrast, or a recorded-covariate standardized contrast is a causal effect merely because it resembles the formula for one.

This distinction also helps separate missing values from inaccurate measurements. A missing value is unavailable; a measured value may instead be present but incorrect. These are different data problems. In either case, first state the causal quantity of interest, then determine how the recorded data relate to it.

Common pitfall: “The outcome is missing for some people, so the causal effect of interest is the effect among people whose outcomes were observed.” This changes the target population without justification. The correction is to state the intended target population and potential-outcome contrast first, then label the responder-only contrast as an observed-data quantity. It can represent the same target only under additional conditions that connect response and the target outcomes.

A precise report can therefore name both quantities. For example: “The target is the population ATE, defined as the mean difference in 12-month outcomes under treatment versus control. The available-record contrast compares observed outcome means among people with recorded treatment and outcome.” This wording does not imply that the second quantity identifies the first.

Check Your Understanding

Use the distinction between the target estimand and the observed-data quantity in each answer.

Define the ATE using potential outcomes and state what the target-population notation refers to.
If an outcome is unrecorded, does its missingness indicator represent an outcome value of zero? Explain.
In the first worked example, calculate the target ATE and the recorded-outcome contrast. Why are these different quantities?
In the covariate example, which distribution supplies the weights for the target ATE, and which distribution supplies the weights for the recorded-subset contrast?
What information would be needed before concluding that an observed-data contrast estimates the target causal effect?