Binary Early Response as a Selection Variable

Why a Binary Early Response Can Guide Selection

In a multi-arm, multi-stage trial, final event outcomes may not yet be sufficiently mature when a decision about continued follow-up is needed. A binary response observed at a prespecified assessment time can provide an interim measure for comparing arms. The aim here is specific: estimate arm-level response rates, contrast each treatment with control, and use the estimates and their uncertainty to inform which arms continue. A response contrast can help rank arms, but it is not automatically a substitute for the final endpoint or proof that one treatment is superior.

A meaningful comparison requires a common response definition and assessment time. For example, the endpoint might be whether a participant has achieved a prespecified improvement by week 8. The response rule should specify how to classify values at the threshold, what to do after treatment discontinuation, and how death or other events before week 8 affect classification. Those choices define the outcome; they should not be made after looking at which choice favours a particular arm.

The target population and analysis set must also be stated. In the usual intention-to-treat interpretation, the probability refers to participants assigned to an arm, regardless of whether they complete treatment. Each participant must have an outcome classified under the prespecified rule. Excluding participants with unavailable assessments can distort the arm-level rates if availability is related to response or treatment. A missing outcome is not automatically a nonresponse: the protocol must specify an appropriate endpoint or missing-data strategy, and the assumptions behind that strategy must be considered.

Estimating a Contrast and Its Uncertainty

Suppose \(n_j\) participants are assigned to arm \(j\), and \(x_j\) of them meet the binary response definition. The observed response rate is \(\hat p_j=x_j/n_j\), and the estimated treatment-versus-control risk difference is \(\hat\Delta_j=\hat p_j-\hat p_0\). Under independent participant outcomes within and between randomized arms, with a common response probability within each arm, the count satisfies \(X_j\sim\operatorname{Binomial}(n_j,p_j)\). The usual estimated standard error for the risk difference is

The variance terms add because the treatment-arm and control-arm samples are independent. A simple approximate 95% confidence interval is \(\hat\Delta_j\pm1.96SE(\hat\Delta_j)\). Its endpoints are in probability-difference units: for example, 0.10 means a difference of 10 percentage points. The interval describes uncertainty in the risk difference under the sampling model; it is not a probability statement about where the fixed parameter lies.

A short derivation explains the standard error. For a Bernoulli outcome with response probability \(p_j\), the sample mean of \(n_j\) independent outcomes has variance \(p_j(1-p_j)/n_j\). Since the treatment and control sample means are independent, the variance of their difference is the sum of their variances:

Replacing the unknown probabilities with the observed response rates gives the estimated variance in the standard-error formula. This approximation is most comfortable when arm sample sizes are not small and response rates are not close to 0 or 1. With sparse counts or boundary rates, Wald intervals can have poor coverage or even extend beyond the possible risk-difference range from -1 to 1. Score-based or suitable binomial intervals are preferable in those settings.

Shared Control and Uncertainty in the Ranking

When several treatments are contrasted with the same control, the estimated contrasts are correlated. In particular, under independent randomized arm samples,

Both differences contain the same estimated control rate with a minus sign, so an unusually high or low control rate shifts all treatment-control contrasts together. This dependence matters when comparing contrasts jointly; it is not correct to treat them as independent simply because the treatment arms differ. To compare two treatments directly, the shared control cancels: \(\hat\Delta_j-\hat\Delta_k=\hat p_j-\hat p_k\). Under independent treatment-arm samples, its estimated standard error is based on those two arms alone.

An uncertainty-aware selection rule should say in advance how estimates and uncertainty will be used. One transparent approach is to rank by the estimated risk difference, report an interval for each contrast, and flag arms whose rankings are not clearly separated. If continued follow-up is limited to a fixed number of arms, the rule can use a prespecified practical threshold or a defined uncertainty criterion, rather than treating tiny differences between noisy estimates as decisive. The criterion should fit the purpose of selection; an interim ranking is not automatically a confirmatory significance test.

Assumptions, Diagnostics, and Alternatives

Before using these summaries for selection, check the design and data that make them meaningful. The assessment time should be the same across arms, and participants should have had a comparable opportunity to reach it. Confirm that the response threshold and rules for discontinuation and events before assessment were applied consistently. Check counts and denominators by arm, the extent and pattern of unavailable outcomes, and whether any data-quality issue could have changed classification. Also check whether the independent-binomial approximation is appropriate: clustering, repeated contributions from the same participant, or other dependence requires a variance method that reflects that structure.

A raw response rate among only those assessed is not necessarily an estimate of the randomized-arm response probability. If the proportion assessed differs by arm, or assessment availability is related to response, the observed proportions may be misleading. Presenting missing participants as nonresponders can be defensible only when that is part of the defined endpoint; it is not a neutral default. The selection decision should acknowledge the analysis strategy and its assumptions rather than hiding the missingness problem in a denominator.

The risk difference is directly interpretable in percentage points, but it is not the only possible effect scale. Risk ratios and odds ratios answer different questions; when treatment arms share a control, risk difference, risk ratio, and odds ratio preserve the same treatment-arm ranking, but rankings can differ when comparing effects across settings or strata with different baseline probabilities. The chosen scale should match the selection objective and be prespecified. For small samples or rates near the boundaries, score-based intervals or other suitable binomial methods are alternatives to the Wald interval. In a design with several treatment comparisons, the contrasts also share control information; methods that jointly account for that dependence are preferable to calculations that assume independent comparisons.

A further limitation is selection on noisy estimates. When the arm with the largest observed response is chosen, random variation tends to make that arm's observed rate look unusually favourable. The introductory material on post-selection estimation addresses the consequences of selecting on observed effects; here the practical safeguard is to preserve uncertainty in the selection summary and avoid presenting the winning point estimate as an unbiased measure of its effect. Finally, a binary response at \(t^*\) is only an intermediate selection measure. Its association with, or ability to predict, a later event outcome must not be assumed from the response-rate comparison alone.