Three Different Things Behind a Treatment-Effect Result
A trial report may state that the treatment group improved by 1.34 points more than the control group. That number is not the treatment-effect question itself, nor is it the general calculation rule. It is one result obtained by applying a procedure to particular data. Keeping these three ideas separate is essential: the estimand states what quantity is targeted, the estimator states how data are used to estimate it, and the estimate is the value produced in the observed trial.
The distinction matters in sensitivity analysis and in interpreting trial results. Two analyses can produce different numerical results because they use different estimators, different assumptions, or even different targets. A clear report identifies which of these changed rather than treating every reported number as an answer to the same question.
An estimand is not itself a calculation procedure. For a randomized trial, it might be the average difference in a specified outcome at a specified time between two treatment conditions in a specified population. The five attributes of a trial estimand specify the question in more detail. Here, assume those attributes have already been chosen; the focus is on separating that target from the data analysis and its result.
A Treatment Contrast and Its Sample-Mean Estimator
Suppose a trial randomizes \(N\) participants to treatment or control, and the outcome is measured at a planned follow-up time. Let \(Y_i(1)\) and \(Y_i(0)\) denote participant \(i\)'s potential outcome under treatment and control, respectively. Only the outcome under the assigned treatment is observed for each participant. For a finite trial population, one possible estimand is the average treatment-minus-control contrast:
This quantity is defined by the population and the two treatment conditions, not by the observed sample means. For example, if higher scores are better, a positive \(\tau\) means that the average potential outcome under treatment exceeds the average potential outcome under control. The units are the units of the outcome.
A simple estimator is the difference between the observed treatment-arm mean and the observed control-arm mean:
Here \(\bar{Y}_{1}\) and \(\bar{Y}_{0}\) are sample means in the two randomized arms. The estimator is a rule: before the trial data are observed, its value is not yet known. Once the data are available, substituting the observed arm means gives an estimate, such as \(-1.34\) points.
Why Randomization Supports This Estimator
For the sample-mean estimator to target the finite-population contrast above in a simple design-based argument, assume that treatment is randomly assigned, the two treatment conditions are well-defined, and the outcome is observed for every randomized participant at the specified time. The potential outcomes are treated as fixed for this argument; randomness comes from the assignment. These assumptions let us compare the estimator’s randomization average with the target.
Under complete randomization, each participant has the same probability of assignment to either arm. The expected treatment-arm mean, averaging over possible random assignments, is therefore the population mean of the treatment potential outcomes. The same reasoning applies to the control arm. Subtracting these expectations gives the justification:
Thus, under these conditions, the difference in arm means is unbiased over the randomization distribution for this estimand. Unbiasedness does not mean that the estimate in any one trial must equal \(\tau\). Nor does randomization by itself resolve what the target should be, whether outcomes are missing, or whether the chosen estimator is appropriate for a more complicated design.
Worked Example: From Target Contrast to Numerical Estimate
Consider a hypothetical randomized trial with a continuous outcome measured in points at the planned follow-up. The target is the average treatment-minus-control contrast in the randomized trial population. Assume treatment was randomized as planned and the outcome was observed for all participants. The treatment arm has 100 participants whose scores sum to 1,433 points; the control arm has 100 participants whose scores sum to 1,567 points.
Estimand. The target is the population-average difference in follow-up score under the two treatment conditions, in points. It is a quantity defined independently of these observed sums.
Estimator. Use the difference in observed arm means, \(\widehat{\tau}=\bar{Y}_{1}-\bar{Y}_{0}\). Its validity for the stated target relies here on the randomization and complete-outcome assumptions just described.
Estimate. The observed means are
Substitution gives
The estimate indicates that the treatment-arm mean was 1.34 points lower than the control-arm mean in this hypothetical trial. It does not establish that the population estimand is exactly \(-1.34\) points; sampling variation means a realized estimate can differ from its target.
Different Estimators and Different Targets
The same estimand can sometimes be estimated by more than one procedure. For example, an unadjusted difference in arm means and a baseline-adjusted regression analysis may both be used to estimate a marginal treatment contrast. Their numerical estimates can differ because they use the observed data differently. To interpret the adjusted result as targeting the same quantity, the model and the way its results are translated to the target population must support that interpretation. A model coefficient should not automatically be assumed to equal the desired population contrast.
Conversely, the same trial data can be used to calculate estimates of different estimands. A contrast in follow-up scores and a contrast in change from baseline are not interchangeable just because both are expressed in points. The estimand must specify which outcome quantity is the target.
Worked Example: Endpoint Contrast Versus Change Contrast
In a hypothetical trial, the treatment-arm mean is 50 points at baseline and 62 points at follow-up. The control-arm mean is 48 points at baseline and 59 points at follow-up. Assume the outcome is observed for all participants, the same population is used for both summaries, and treatment was randomized. Compare two possible targets, both expressed as treatment minus control.
Endpoint target. If the target is the difference in mean follow-up scores, the sample-mean estimator gives
Change target. If the target is the difference in mean changes from baseline, the treatment-arm change is \(62-50=12\) points and the control-arm change is \(59-48=11\) points. The estimate for this target is therefore
The estimates, 3 points and 1 point, differ because they answer different questions. Neither number can be substituted for the other without changing the target. The baseline values also differ between arms, so the arithmetic alone does not establish which target is scientifically preferable; that choice belongs in the estimand specification.
A useful comparison is to ask, in order: What population quantity is sought? What data-based rule is being used? What value did that rule produce? If two procedures are proposed as alternatives, also ask whether they target the same quantity and what assumptions justify each procedure.
Reading Results and Diagnosing Mismatches
A reported estimate should be accompanied by enough information to connect it to its estimand and estimator. State the outcome and units, the treatment contrast direction, the target population and time point, and the procedure used. Uncertainty summaries such as confidence intervals describe uncertainty associated with an estimator under its assumptions; they do not turn the estimate into the estimand or redefine the target.
Several warning signs suggest that the three concepts have been blurred:
- The target is described only as a method. “The estimand is the difference in sample means” confuses the population quantity with one estimator for it.
- The estimate is treated as the true target. A value such as \(-1.34\) points is an observed result, not proof that the population contrast equals exactly that amount.
- The direction or units are omitted. A contrast of treatment minus control has the opposite sign from control minus treatment. A number without the outcome units is incomplete.
- Different analyses are compared as if they necessarily answer the same question. First check whether they use the same outcome definition, population, time point, and treatment contrast.
- The procedure assumes data that are not available. The simple derivation above assumes complete outcome measurement. If outcomes are missing, that derivation does not by itself justify using observed arm means for the original target; additional assumptions and an appropriate procedure are needed.
A practical diagnostic is to write one sentence for each item before interpreting results: “The estimand is …”; “The estimator is …”; “The estimate is …”. If the first sentence contains a sample statistic, or the second is just a numerical value, revise the description. In sensitivity analysis, this separation is especially important: a changed result may reflect a changed estimate, a changed procedure, or a changed target, and those have different interpretations.
Check Your Understanding
For each question, identify the relevant target, procedure, or result. Solutions follow each question.
- A randomized trial targets the average treatment-minus-control difference in a 10-week outcome, in points. The chosen procedure is the difference in observed arm means, and the result is 2.4 points. Which item is the estimate, and what is its direction?
Solution: The estimate is 2.4 points. Because the contrast is treatment minus control, the observed treatment-arm mean is 2.4 points higher than the control-arm mean. - In a hypothetical trial, the treatment-arm mean is 21.6 points and the control-arm mean is 20.1 points. Using treatment minus control, calculate the estimate from the difference-in-means estimator.
Solution: Substitute the means: \(21.6-20.1=1.5\) points. The estimate is 1.5 points, not the estimand itself. - A report gives both a follow-up-score contrast and a change-from-baseline contrast. Are these necessarily two estimates of the same estimand?
Solution: No. The follow-up contrast targets a difference in follow-up scores, while the change contrast targets a difference in changes. They can be estimates of distinct estimands, even if both use the same trial participants and units. - Why does randomization not guarantee that a single observed difference in arm means equals the finite-population treatment contrast?
Solution: Randomization supports unbiasedness over the distribution of possible assignments under the stated assumptions. A particular assignment can produce sampling variation, so its estimate need not equal the target exactly. - A trial has missing follow-up outcomes, but an analyst applies the observed difference in arm means and labels it unbiased for the target. What is the issue?
Solution: The complete-outcome condition used in the simple randomization argument is not met. Randomization alone does not establish that the observed means estimate the original target when outcomes are missing; assumptions and a procedure addressing the missing outcomes are required.