Why the Outcome Family Matters
In a time-to-event multi-arm multi-stage trial, an interim selection decision may be made before enough final events have accumulated to compare arms on the primary endpoint. The selection variable is the interim measurement used to rank or screen the experimental arms. Its family determines what is measured, how an arm contrast is calculated, and which assumptions are needed for the contrast to be useful at that look.
Four broad families arise frequently: binary response, continuous biomarker, repeated-measure outcome, and early time-to-event outcome. They can all be used to construct selection evidence, but they are not interchangeable. A binary response may be simple to interpret but discards variation within response categories. A continuous biomarker preserves numerical differences but may not be clinically meaningful. Repeated measures can use change over time, but require a consistent measurement schedule. Early event outcomes are directly on the event-time scale, but may be immature or based on few events.
For arm \(k\), let \(\hat{\Delta}_k\) denote its estimated selection contrast and let \(SE(\hat{\Delta}_k)\) denote its standard error. A common way to put the contrast on a dimensionless scale is
This score is one possible selection statistic, not a universal requirement. A protocol might instead rank arms by a clinically scaled contrast or apply a prespecified threshold. The contrast, its direction, and the selection rule must be specified before the interim data are examined. A favorable score is evidence for ranking under that rule; it does not by itself establish benefit on the final time-to-event endpoint.
Four Outcome Families and Their Summaries
A binary response records whether each participant meets a defined response criterion by a specified time. If higher response probability is favorable, one simple contrast is the difference in response proportions, \(\hat{p}_k-\hat{p}_0\), where \(k\) denotes an experimental arm and \(0\) denotes control. This summary is straightforward when the response definition and assessment time are uniform. Its weaknesses include sensitivity to the cutoff used to define response and loss of information when participants just above and below the cutoff are treated as different while participants with very different values on the same side are treated alike.
A continuous biomarker is summarized using a numerical contrast, often a difference in means at a prespecified assessment time or in change from baseline. Its units remain interpretable, and differences across the measurement range are retained. The measurement must be sufficiently reliable and available by the selection look. A numerical difference is not automatically clinically meaningful: its scale, direction, and relevant range need substantive justification. Extreme values or skewness can also make a mean and its standard error unstable.
A repeated-measure outcome consists of measurements on the same participant at multiple planned times. A selection summary could be change from baseline to a prespecified visit, or a prespecified average over visits. Repeated observations from one participant are not independent observations. A summary based on change must therefore use the variability of participant-level changes, which already reflects the within-person association between baseline and follow-up. Choosing a visit or summary after seeing which gives the best arm ranking would make the selection rule data-dependent.
An early time-to-event outcome records event times and, when applicable, censoring times before the final endpoint has matured. It may be summarized by event-free status at a fixed landmark, or by another prespecified summary that uses event times. Event-free status at a landmark is easy to explain but loses information about when events occurred before that point. Analyses that use event times must account for censoring appropriately; incomplete follow-up cannot simply be treated as event-free follow-up.
| Family | Example selection summary | Practical strength | Common limitation |
|---|---|---|---|
| Binary response | Difference in response proportions by a fixed time | Directly interpretable as a proportion difference | Cutoff dependence and discarded within-category information |
| Continuous biomarker | Difference in mean value or mean change | Retains numerical variation and units | May be noisy, skewed, or weakly meaningful clinically |
| Repeated measure | Difference in change at a prespecified visit | Can capture an evolving response | Requires consistent visits and appropriate participant-level variation |
| Early event outcome | Difference in event-free probability at a landmark | Directly reflects early events on the event-time scale | May be immature; landmark status discards event timing |
Worked Comparisons
Worked Example: Binary Response at an Interim Look
Suppose a hypothetical trial has 60 participants in an experimental arm and 60 in control with response status available by the prespecified interim time. There are 42 responders in the experimental arm and 30 in control. Assume response is defined identically in both arms, all 120 statuses are observed, and participants are independent across arms. For this simple pairwise comparison, estimate the response-proportion difference and its usual independent-binomial standard error.
The proportions are \(\hat{p}_k=42/60=0.700\) and \(\hat{p}_0=30/60=0.500\). With higher response probability prespecified as favorable,
The estimated standard error is
Thus \(Z_k^{\mathrm{sel}}=0.200/0.08756=2.284\). The observed response proportion is 20.0 percentage points higher in the experimental arm, and the standardized selection score is positive in the favorable direction. This is an interim ranking summary, not a claim that the final event endpoint will improve by 20.0 percentage points.
Worked Example: Continuous Biomarker Contrast
In a hypothetical interim comparison, a biomarker is measured in the same units for 25 participants per arm. The experimental-arm mean is 8.4 units with sample standard deviation 3.0 units; the control mean is 6.9 units with sample standard deviation 3.3 units. Suppose larger values are favorable, the arms are independent, and the sample means and standard deviations are adequate summaries for this comparison.
The estimated contrast is \(8.4-6.9=1.5\) biomarker units. The estimated standard error is
The selection score is \(1.5/0.8920=1.682\). The experimental-arm mean is higher by an estimated 1.5 units, with a positive standardized score under the stipulated favorable direction. Before using this summary, investigators would need to check that the units are comparable across arms, that the biomarker is not dominated by extreme observations, and that the assessment was available under the planned interim schedule.
Worked Example: Prespecified Repeated-Measure Change
In a hypothetical trial, a biomarker is measured at baseline and week 8. Lower values are favorable. For 30 participants per arm, the mean week-8 minus baseline change is \(-4.2\) units in the experimental arm and \(-2.5\) units in control. The sample standard deviations of participant-level changes are 4.0 and 3.5 units, respectively. Assume the week-8 visit and change summary were prespecified, changes are observed for all participants, and participants are independent across arms.
Because a more negative change is favorable, define the oriented contrast as control change minus experimental change:
Using the standard deviations of the participant-level changes, not separate baseline and week-8 standard deviations, gives
Therefore \(Z_k^{\mathrm{sel}}=1.7/0.9704=1.752\). The estimated biomarker reduction is 1.7 units greater in the experimental arm, in the favorable direction. This calculation uses one prespecified summary per participant; it does not treat the two measurements on each participant as independent.
Worked Example: Event-Free Status at a Fixed Landmark
Consider a hypothetical interim readout at a prespecified landmark time \(\tau\). Among 50 participants per arm, 45 in the experimental arm and 40 in control are known to be event-free at \(\tau\). Assume every participant has follow-up through \(\tau\), the event definition is consistent across arms, and the arms are independent. Higher event-free probability is favorable.
The estimated event-free proportions are \(45/50=0.900\) and \(40/50=0.800\), so the oriented difference is \(0.900-0.800=0.100\). Its estimated standard error is
The selection score is \(0.100/0.07071=1.414\). Event-free probability is estimated to be 10.0 percentage points higher in the experimental arm at \(\tau\). This landmark summary does not distinguish an event just before \(\tau\) from one much earlier, and the calculation would not be justified if participants censored before \(\tau\) were incorrectly counted as event-free.
Justifying a Standardized Contrast
The standardization in the selection score is useful because the raw contrasts above have different units: a proportion difference, biomarker units, and event-free probability. The reason for dividing by the standard error follows from the variance of a difference. For independent arm-specific estimates with variances \(V_k\) and \(V_0\), the variance of their difference is
because the covariance is zero under independence. The standard error estimates the square root of this variance. Dividing the oriented contrast by that standard error measures its size relative to its estimated sampling variability. A larger score can therefore reflect a larger estimated contrast, greater precision, or both. This explains why standardized ranking and ranking by raw clinical effect need not agree.
In a multi-arm design, experimental-arm estimates may share a control group, so their selection scores can be statistically dependent. The pairwise calculations above do not quantify that cross-arm dependence. The selection rule should use the design’s intended comparison structure, and its operating characteristics should be checked under that structure rather than inferred from separate arm-by-arm calculations.
Diagnostics, Alternatives, and Limits
The main alternatives are to select using a clinically scaled raw contrast, a standardized contrast, or a prespecified threshold applied to one of these summaries. A raw contrast makes the magnitude immediately interpretable but can be difficult to compare when precision differs substantially across arms. A standardized contrast puts evidence on a common scale but can favor a less precise or more precise arm differently from a clinical-effect ranking. A threshold can prevent selection based on trivial differences, but its value must be justified in advance. None of these choices removes the need to define the outcome and its timing consistently.
Useful diagnostics focus on whether the planned summary is being estimated as intended:
- Check the number and proportion of participants with an observed outcome in each arm, and compare missingness patterns across arms.
- For continuous measurements, inspect the distribution, range, and influential observations; verify units and measurement procedures.
- For repeated measures, confirm that visit windows and measurement schedules are comparable, and use the variability of the chosen participant-level summary.
- For early event outcomes, tabulate events and follow-up maturity; verify that censoring has not been mistaken for absence of an event.
- Confirm that the favorable direction and any response cutoffs were fixed before looking at arm results.
The calculations also rely on assumptions that may fail in practice. Outcome definitions can be applied inconsistently; measurement error can obscure biomarker differences; missingness can make observed participants unrepresentative; and an early event summary can be unstable when few events have occurred. A selection variable can be operationally available yet poorly aligned with the final scientific objective. Selection based on a noisy interim contrast can also rank arms differently from their underlying effects. These are reasons to evaluate the prespecified selection procedure, not reasons to reinterpret a favorable interim score as proof of final benefit.
Define the outcome, assessment time or summary, favorable direction, and handling of missing or censored data before using interim results.
Use a summary and standard error suited to the outcome family and the planned comparison.
Review data availability and assumptions, then use the prespecified ranking or threshold without treating selection as final efficacy evidence.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- A binary response is defined using a threshold on a continuous biomarker. Name one information loss caused by this choice.
- For a repeated-measure selection summary based on change, which standard deviation should enter the two-arm standard error?
- Why can ranking arms by standardized contrast produce a different order from ranking by raw contrast?
- At a landmark, a participant is censored before the landmark without a recorded event. Can that participant automatically be counted as event-free at the landmark?
- An experimental arm has the highest estimated response proportion. Does that alone show that it should be selected?
Solutions. (1) Values on the same side of the cutoff are treated alike, even if their biomarker values differ substantially. (2) Use the standard deviation of the participant-level changes in each arm, which incorporates within-person variation in change. (3) Standardization also depends on the estimated standard error, so differing precision can change the ranking. (4) No; censoring before the landmark does not establish event-free status through that time. (5) No; the prespecified rule, precision, data availability, and assumptions must also be considered.