Potential Outcomes And Sharp Null Hypotheses

Why Potential Outcomes Matter

In a two-arm trial, each participant receives either treatment or control, and only the outcome under the assigned condition is observed. Yet a treatment-effect question compares what would happen to the same participants under both conditions. Potential outcomes give a precise notation for that comparison: they define the two outcomes that could occur for each participant, while making clear that one is unobserved.

This framework is useful for randomization-based inference because it separates the outcomes of interest from the assignment process. The potential outcomes describe the trial participants’ responses to the treatment conditions; the assignment determines which of those outcomes is observed. As in the earlier material on inferential targets under response-adaptive assignment, the target must be stated before choosing an inferential procedure. Here the focus is the outcome schedule specified by a sharp null hypothesis.

Definition and Assumptions

Definition: For participant $i$, let $Y_i(1)$ be the outcome that would be observed if the participant were assigned to treatment, and let $Y_i(0)$ be the outcome that would be observed if the participant were assigned to control. Let $Z_i=1$ indicate treatment assignment and $Z_i=0$ indicate control assignment. The observed outcome is $$ Y_i^{\mathrm{obs}}=Z_iY_i(1)+(1-Z_i)Y_i(0). $$ The unassigned potential outcome is unobserved.

The notation uses assignment to define the treatment condition. If a participant is assigned treatment but does not take it, $Y_i(1)$ still denotes the outcome under assignment to treatment when the study is analyzed according to assignment. Defining potential outcomes by treatment actually received instead would require a different framework and additional assumptions.

The basic setup assumes that treatment and control are sufficiently well-defined for their potential outcomes to have meaning. It also assumes no interference between participants: one participant’s outcome under an assignment does not depend on the assignments given to other participants. The consistency relation above connects the assigned condition to the observed outcome. These conditions are often summarized as part of the stable unit treatment value assumption, or SUTVA. If treatment versions differ materially, or one participant’s treatment changes another participant’s outcome, the two-potential-outcomes notation may be inadequate without refinement.

In a finite-population randomization perspective, the potential outcomes are treated as fixed attributes of the participants in the trial. Randomness comes from assignment, not from repeatedly sampling new potential outcomes for the same participants. This distinction does not mean outcomes are known: for each participant, one potential outcome remains missing.

Fisher’s Sharp Null

Definition: Fisher’s sharp null hypothesis of no treatment effect is $$ H_0^{F}: Y_i(1)=Y_i(0)\quad\text{for every participant }i. $$ It asserts that treatment assignment would change the outcome of no participant.

The hypothesis is called sharp because it specifies the missing potential outcome for every participant once the observed outcome and assignment are known. Under $H_0^{F}$, write the common value $Y_i(1)=Y_i(0)=y_i$. Whichever condition participant $i$ received, the observed outcome equals $y_i$. Therefore the unobserved potential outcome must also equal the observed one.

H_0^{F} \ \Longrightarrow\ Y_i(1)=Y_i(0)=Y_i^{\mathrm{obs}} \quad\text{for each }i.

More generally, suppose we consider any possible assignment vector $z=(z_1,\ldots,z_n)$, where each $z_i$ is 0 or 1. Under this null, the outcome that would be observed for participant $i$ under $z$ is $$ z_iY_i(1)+(1-z_i)Y_i(0)=z_i y_i+(1-z_i)y_i=y_i. $$ The observed outcome vector is therefore fully determined under the null for every possible assignment: it remains the same vector of participant-level outcomes. This is an imputation under a stated hypothesis, not a prediction based on a fitted outcome model.

Worked Example: Imputing Outcomes Under Fisher’s Null

Suppose four participants in a hypothetical trial have symptom scores measured in points, with lower scores indicating fewer symptoms. Participants 1 and 3 are assigned treatment; participants 2 and 4 are assigned control.

Participant	Assignment $Z_i$	Observed score $Y_i^{\mathrm{obs}}$
1	1	8
2	0	6
3	1	10
4	0	7

Under $H_0^{F}$, the two potential scores for each participant are equal. For participant 1, treatment was observed, so $Y_1(1)=8$; the null implies $Y_1(0)=8$. For participant 2, control was observed, so $Y_2(0)=6$, and the null implies $Y_2(1)=6$. Applying the same reasoning to participants 3 and 4 gives $Y_3(1)=Y_3(0)=10$ and $Y_4(1)=Y_4(0)=7$.

Now consider the alternative assignment $z=(0,1,0,1)$. The imputed outcome for participant 1 is $Y_1(0)=8$; for participant 2 it is $Y_2(1)=6$; for participant 3 it is $Y_3(0)=10$; and for participant 4 it is $Y_4(1)=7$. The outcome vector under that assignment is therefore $(8,6,10,7)$, the same participant-level vector as under the observed assignment. The imputation follows from the sharp null, not from assuming that participants have equal scores to one another.

A Short Justification of the Imputation Rule

The rule can be derived directly from consistency and the null. If $Z_i=1$, consistency gives $Y_i^{\mathrm{obs}}=Y_i(1)$. The null gives $Y_i(0)=Y_i(1)$. Combining these equalities yields $Y_i(0)=Y_i^{\mathrm{obs}}$, so both potential outcomes are known under the null. If $Z_i=0$, consistency gives $Y_i^{\mathrm{obs}}=Y_i(0)$, and the null then gives $Y_i(1)=Y_i^{\mathrm{obs}}$. Thus in either assignment group the missing potential outcome is the observed outcome:

\begin{aligned} Z_i=1 &: \quad Y_i(1)=Y_i^{\mathrm{obs}},\quad Y_i(0)=Y_i(1),\\ Z_i=0 &: \quad Y_i(0)=Y_i^{\mathrm{obs}},\quad Y_i(1)=Y_i(0). \end{aligned}

No averaging or distributional assumption is needed for this deduction. Its key premise is the participant-by-participant equality in $H_0^{F}$; a statement only about an overall average would not, by itself, reveal each missing outcome.

Worked Example: The Imputation for Each Participant

In a hypothetical study of a mobility program, the outcome is walking distance in metres. Three participants have observed data $(Z_i,Y_i^{\mathrm{obs}})$ equal to $(1,420)$, $(0,310)$, and $(1,500)$. Under Fisher’s sharp null:

For participant 1, $Z_1=1$, so $Y_1(1)=420$ m. The null sets $Y_1(0)=Y_1(1)=420$ m.

For participant 2, $Z_2=0$, so $Y_2(0)=310$ m. The null sets $Y_2(1)=Y_2(0)=310$ m.

For participant 3, $Z_3=1$, so $Y_3(1)=500$ m and $Y_3(0)=500$ m. Thus the complete potential-outcome schedule under the null is $(420,420)$, $(310,310)$, and $(500,500)$, where each pair lists control then treatment. The units remain metres in both potential outcomes.

The scores need not be alike across participants. The null says each participant’s own two potential outcomes coincide; it does not say all participants would have the same outcome.

Other Sharp Nulls and What They Specify

Fisher’s null is one member of a broader class. A hypothesis is sharp when it specifies enough about the potential outcomes to determine the missing values for every participant. For example, a constant additive-effect hypothesis fixes a value $\tau_0$ and asserts $Y_i(1)-Y_i(0)=\tau_0$ for each $i$. If $\tau_0$ is not zero, this is not Fisher’s no-effect null, but it is also sharp.

Worked Example: Imputing Under a Constant-Effect Sharp Null

Suppose the outcome is a symptom score in points and a hypothetical sharp hypothesis states that treatment lowers each participant’s score by exactly 2 points: $Y_i(1)-Y_i(0)=-2$. A treated participant has observed score 5. Since $Y_i(1)=5$, the hypothesis gives $5-Y_i(0)=-2$, so $Y_i(0)=7$ points. A control participant has observed score 8. Since $Y_i(0)=8$, the hypothesis gives $Y_i(1)-8=-2$, so $Y_i(1)=6$ points.

The two imputations differ from the Fisher-null rule because the stated sharp hypothesis specifies a nonzero effect. The arithmetic uses the same principle: combine the observed potential outcome with the hypothesis to solve for the unobserved one. This example illustrates imputation only; it does not turn the constant-effect assumption into a claim that the assumption is empirically verified.

Diagnostics, Comparisons, and Limitations

Potential outcomes are not all observed, so the sharp null cannot be checked by directly comparing both outcomes for each participant. The null is a hypothesis used to define what the unobserved outcomes would have to be, not a fact established by seeing the observed data. A useful diagnostic is therefore to check whether the assumptions and data definitions are coherent before using the imputation: Is the endpoint defined in the same way under treatment and control? Is assignment distinguished from treatment received? Could one participant’s assignment affect another participant’s outcome? Are outcomes missing for reasons not represented by the treatment assignment?

A common error is to say, “The treated participants’ missing control outcomes are the control-group mean under the null.” That is not what Fisher’s sharp null says. It equates each participant’s two potential outcomes, so a treated participant with observed outcome 8 has imputed control outcome 8, not the mean among controls. The correction is to perform the imputation participant by participant.

This framework differs from outcome-model approaches that estimate unobserved outcomes using covariates and a fitted relationship. Such models can be useful for other questions, but their predicted values do not follow solely from Fisher’s null. Conversely, the sharp-null imputation is exact only conditional on the sharp hypothesis and the stated potential-outcome assumptions; it does not supply plausible missing outcomes when the hypothesis is false.

Care is also needed when outcomes are missing because of dropout or measurement failure. If a participant’s assigned-condition outcome itself was not observed, consistency alone does not provide that value. Fisher’s null equates the two potential outcomes, but does not automatically recover an outcome that is missing under both conditions. A separate missing-data assumption or procedure would be needed. Likewise, noncompliance does not change the meaning of $Y_i(1)$ if it is defined by assignment; changing the definition to treatment received changes the question and may require further assumptions.

Key takeaway: Under Fisher’s sharp no-effect null, each participant’s missing potential outcome equals that participant’s observed outcome. This identifies the full outcome schedule under every assignment, but only because the null makes a claim about every individual.

Check Your Understanding

Answer each question, then compare with the solutions.

A participant assigned to control has an observed outcome of 12 units. Under Fisher’s sharp null, what are $Y_i(0)$ and $Y_i(1)$?
Explain why a treated participant’s missing control outcome is not generally the observed control-group mean under Fisher’s null.
Under the sharp hypothesis $Y_i(1)-Y_i(0)=3$, a control participant has observed outcome 9 units. Impute the treatment potential outcome.
Does Fisher’s sharp null imply that all participants have the same outcome? Explain.

Solutions. 1. Since the participant was assigned control, $Y_i(0)=12$. Fisher’s null gives $Y_i(1)=Y_i(0)=12$ units. 2. The null equates the two outcomes within each participant; it does not equate an individual outcome to a group average. If that treated participant’s observed outcome is $y$, the missing control outcome is $y$. 3. Here $Y_i(0)=9$ and $Y_i(1)-9=3$, so $Y_i(1)=12$ units. 4. No. It requires $Y_i(1)=Y_i(0)$ for each participant, but the common value can differ between participants.