Median-Unbiased Estimation After Arm Selection

Why the Conditional Median Is a Different Target

When an arm is selected because its interim estimate is large, the estimate for that arm is generally unusually high relative to its treatment effect. A correction based on the expected selection bias subtracts an estimate of the conditional mean excess. But the conditional mean is only one feature of the selected estimate’s distribution. If that distribution is asymmetric, matching its mean does not make an estimate median-unbiased.

Median-unbiased estimation asks a different question: for a proposed treatment effect, is the observed estimate above or below the median of its distribution given the selection event? The answer is obtained by inverting the conditional distribution. This makes the selection event part of the estimand procedure, not merely a reason to apply a generic downward adjustment.

Definition: Let $Y$ be an estimate of a scalar treatment effect $\theta$, and let $S$ be a specified selection event with $P_\theta(S)>0$. A conditional median-unbiased estimate $\hat{\theta}_{\mathrm{med}}$, for observed $Y=y$ and $S$, solves $$ P_{\theta}\!\left(Y\le y\mid S\right)=\frac{1}{2} $$ with $\theta=\hat{\theta}_{\mathrm{med}}$. This definition presumes the solution exists and is unique. In a regular scalar model where the conditional distribution is continuous and shifts monotonically with $\theta$, the resulting estimator has conditional median-unbiasedness: under the true $\theta$, $P_\theta(\hat{\theta}_{\mathrm{med}}\le\theta\mid S)=1/2$.

For a MAMS design, $S$ must be stated precisely. It could mean that a particular arm was the interim winner, that it passed a continuation rule, or that a complete sequence of interim decisions occurred. Conditioning on the winner alone and conditioning on the full selection path are generally different, because they describe different conditional distributions. The earlier material on conditioning on the winner or the full selection path motivates this distinction.

Deriving the Conditional-Median Equation

Suppose an estimate $Y$ follows a normal sampling model $Y\sim N(\theta,\sigma^2)$, with known standard deviation $\sigma$, and is observed only when it exceeds a threshold $c$. This is a simple model of selection by an efficacy boundary. The model assumes that, before selection, $Y$ is unbiased for $\theta$, its variance is $\sigma^2$, and the stated threshold fully describes the selection event.

For $y>c$, the conditional cumulative distribution function is the probability of being both selected and no greater than $y$, divided by the probability of selection. Standardizing the normal variable gives

F_\theta(y\mid Y>c) = \frac{P_\theta(c<Y\le y)}{P_\theta(Y>c)} = \frac{\Phi\!\left((y-\theta)/\sigma\right)-\Phi\!\left((c-\theta)/\sigma\right)} {1-\Phi\!\left((c-\theta)/\sigma\right)}.

The median-unbiased estimate is the value of $\theta$ that makes this conditional probability equal to $1/2$. In practice, one solves the resulting equation numerically when it has no closed-form solution. The conditional mean follows a different equation. For this one-sided truncated normal model,

E_\theta(Y\mid Y>c) = \theta+\sigma\frac{\phi(a)}{1-\Phi(a)}, \qquad a=\frac{c-\theta}{\sigma},

where $\phi$ and $\Phi$ are the standard normal density and cumulative distribution functions. A conditional-mean correction estimates $\theta$ by matching this expectation to $y$. The median procedure instead matches the conditional median to $y$. Unless the conditional distribution is symmetric about its mean, these are not equivalent operations.

Worked Example: A Winner Selected Above Zero

Suppose a hypothetical arm’s estimate has model $Y\sim N(\theta,1)$, with the effect and standard deviation both measured in effect units. The arm is retained only if $Y>0$, and its observed estimate is $y=1.000$. Assume the normal model and threshold are known, and that there is no additional selection rule.

The selection probability is $P_\theta(Y>0)=\Phi(\theta)$. Therefore the conditional CDF at the observation is $$ P_\theta(Y\le 1\mid Y>0) = \frac{\Phi(1-\theta)-\Phi(-\theta)}{\Phi(\theta)}. $$ Setting this equal to $1/2$, and using $\Phi(-\theta)=1-\Phi(\theta)$, gives $$ \Phi(1-\theta)=1-\frac{1}{2}\Phi(\theta). $$ Numerically solving this equation gives $\hat{\theta}_{\mathrm{med}}\approx 0.686$ effect units. At this value, the observed estimate is the median of the distribution of $Y$ conditional on passing the threshold.

Now compare the conditional-mean correction. Here $$ E_\theta(Y\mid Y>0)=\theta+\frac{\phi(\theta)}{\Phi(\theta)}. $$ Matching this conditional mean to the observed $y=1.000$ requires solving $$ \theta+\frac{\phi(\theta)}{\Phi(\theta)}=1.000. $$ The numerical solution is approximately $\theta=0.481$ effect units. Substitution using unrounded normal-function values gives $$ 0.481+\frac{\phi(0.481)}{\Phi(0.481)}\approx 1.000 $$ effect units. Thus the conditional-mean correction and the conditional-median estimate differ: approximately $0.481$ versus $0.686$ effect units. Both account for the threshold, but they target distinct features of the selected distribution.

Thresholds and More General Selection Events

The threshold example is transparent, but selection in a MAMS design can depend on several interim contrasts, their standard errors, and the decisions at earlier stages. In that setting, write $Y_j$ for the estimate of the effect of arm $j$, and $S_j$ for the event that arm $j$ is selected under the specified rule. The conditional-median estimate solves

P_{\boldsymbol{\theta}}\!\left(Y_j\le y_j\mid S_j\right)=\frac{1}{2}, \qquad \theta_j=\hat{\theta}_{j,\mathrm{med}}.

The probability is computed under the joint sampling model for the estimates and the precise selection rule. In particular, shared-control contrasts can be correlated, and arm rankings can depend on unequal standard errors. Treating contrasts as independent or replacing the actual selection event with a simple threshold can therefore give the wrong conditional CDF. If other treatment effects or covariance parameters are unknown, the conditional distribution also depends on them; treating them as known or plugging in estimates requires an explicit justification and can change the procedure’s finite-sample properties.

Worked Example: A Nonzero Continuation Threshold

Consider a hypothetical estimate $Y\sim N(\theta,1)$, retained only if $Y>0.500$ effect units. Suppose the observed estimate is $y=1.500$ effect units. The assumptions are a normal estimate with known standard deviation $1$ effect unit and a selection rule determined solely by this threshold.

The conditional median equation is $$ \frac{\Phi(1.500-\theta)-\Phi(0.500-\theta)} {1-\Phi(0.500-\theta)} =\frac{1}{2}. $$ Rearranging gives $$ \Phi(1.500-\theta) = \frac{1+\Phi(0.500-\theta)}{2}. $$ Solving this equation numerically yields $\hat{\theta}_{\mathrm{med}}\approx 1.186$ effect units. The value is below the observed estimate because the estimate had to exceed the continuation threshold to be observed in this selected set. The calculation uses the actual threshold of $0.500$, rather than assuming that selection was at zero.

This example also shows why the event must be recorded correctly. If the true rule had included another interim condition, the displayed conditional CDF would not describe the relevant selection distribution; the equation would have to condition on that full event instead.

Why the Inversion Gives a Conditional Median

The key justification is a quantile-inversion argument. Fix a true parameter value $\theta$ and condition on $S$. Suppose the conditional CDF $F_\theta(y\mid S)$ is continuous and strictly increasing in $y$. Let $\hat{\theta}_{\mathrm{med}}$ be the unique solution of $F_{\hat{\theta}_{\mathrm{med}}}(Y\mid S)=1/2$. Also suppose that, at fixed $y$, $F_\theta(y\mid S)$ decreases as $\theta$ increases. These monotonicity conditions hold in the one-sided normal example and must be checked or justified for a more complicated selection model.

At the true $\theta$, the event that the estimate is no greater than the true parameter’s conditional median is

\{\hat{\theta}_{\mathrm{med}}\le\theta\} = \{F_\theta(Y\mid S)\le 1/2\}.

The equality follows from the assumed decreasing relationship between $F_\theta(y\mid S)$ and $\theta$: the solution to $F_{\hat{\theta}_{\mathrm{med}}}(Y\mid S)=1/2$ is at most $\theta$ exactly when $F_\theta(Y\mid S)\le1/2$. Under the conditional distribution given $S$, a continuous CDF evaluated at its own random draw is uniform on $(0,1)$. Consequently,

P_\theta(\hat{\theta}_{\mathrm{med}}\le\theta\mid S) = P_\theta\!\left(F_\theta(Y\mid S)\le\frac{1}{2}\mid S\right) = \frac{1}{2}.

This proves conditional median-unbiasedness under the stated conditions. If the conditional distribution has atoms, the CDF is not continuous and an exact equality of one half may not be attainable; randomized conventions or a median defined through inequalities may then be needed. If monotonicity or uniqueness fails, a numerical root alone does not establish the claimed property.

Worked Example: Independent Later-Stage Data

Suppose a hypothetical MAMS trial uses stage-one data to select an arm, and the selected arm’s stage-two estimate is based on new participants independent of all stage-one data. Conditional on the selected arm and its treatment effect $\theta_j$, assume the stage-two estimate $Y_{2j}\sim N(\theta_j,0.8^2)$. The observed stage-two estimate is $2.400$ effect units. The selection event $S_j$ is determined only by stage-one data, and the stage-two model has the same target effect.

Independence means conditioning on $S_j$ does not change the stage-two sampling distribution: $$ P_{\theta_j}(Y_{2j}\le y\mid S_j) = \Phi\!\left(\frac{y-\theta_j}{0.8}\right). $$ The median equation at the observation is $$ \Phi\!\left(\frac{2.400-\theta_j}{0.8}\right)=\frac{1}{2}. $$ Since $\Phi(0)=1/2$, the solution is $\hat{\theta}_{j,\mathrm{med}}=2.400$ effect units. No selection correction is needed for this stage-two estimate conditional on the stated event and independence assumptions. This conclusion would not automatically apply to a combined estimate that reuses the selected stage-one data.

Assumptions, Diagnostics, and Alternatives

A conditional-median estimate is only as appropriate as its conditional model. Before solving its defining equation, specify the target arm and effect scale, the estimate being inverted, and the event $S$. Then verify that the sampling distribution and its dependence structure match the design. In MAMS applications, important checks include whether contrasts share a control, whether variances differ by arm, whether the selection rule uses standardized statistics or raw estimates, and whether the event includes all relevant interim decisions.

Numerical diagnostics matter too. Evaluate the conditional CDF over a grid of candidate effect values, check that the root is bracketed, and verify that it is monotone across the bracket. Substitute the reported root back into the defining equation and confirm that the conditional probability is close to $1/2$. If simulation is used to approximate the conditional probability, inspect Monte Carlo error and ensure simulated trials are conditioned on the same selection event as the observed trial.

The main alternative considered here is conditional-mean correction. It targets $E_\theta(Y\mid S)$, while median-unbiased inversion targets the conditional median. The former can be useful when expected error is the relevant criterion; the latter gives a median-calibrated estimate under its assumptions. Neither criterion guarantees that every selected estimate is close to the true effect. A third perspective is to report an unadjusted estimate while explicitly acknowledging selection. That avoids model-based correction but does not remove selection bias. These procedures answer different questions, and none should be described as universally best.

Common pitfall: Subtracting the conditional expected bias and then calling the result “median-unbiased” is generally incorrect. That subtraction calibrates a mean, not a median. In the one-sided normal example, the mean-matched estimate is about $0.481$, whereas solving the conditional-median equation gives about $0.686$ effect units. Use the label “median-unbiased” only when the appropriate conditional median equation has been solved and its model assumptions are defensible.

Practical Workflow

Define the target and selection event.
State which arm’s effect is being estimated and whether conditioning is on the winner, a continuation decision, or the full selection path.

Specify the joint sampling model.
Include the estimate’s variance and any dependence created by shared controls or repeated stages; state which nuisance quantities are treated as known.

Construct the conditional CDF.
Compute the probability of observing an estimate no greater than the observed value and divide by the probability of the stated selection event.

Invert and check.
Solve for the effect making the conditional CDF one half, then check the root, monotonicity, and sensitivity to modeling choices.

The central lesson is that selection changes the distribution on which an estimate is judged. Conditional-median estimation responds by matching the observed value to the median of that selected distribution. Its result is not simply a renamed mean correction: the distinction is consequential whenever selection produces skewness, truncation, or a more complex conditional shape.

Check Your Understanding

Answer each question, then compare with the brief solutions.

In the definition of a conditional median-unbiased estimate, what probability is set equal to one half?
Why can a conditional-mean correction differ from a conditional-median estimate after selection?
In the independent later-stage example, why did conditioning on stage-one selection leave the stage-two estimate’s distribution unchanged?
What must be checked before interpreting a numerical solution as a conditional median-unbiased estimate?

Solutions. (1) The conditional probability $P_\theta(Y\le y\mid S)$, evaluated at the candidate effect equal to the estimate, is set to $1/2$. (2) The conditional mean and median are different features of a generally asymmetric selected distribution. (3) The stage-two data were assumed independent of stage-one data, and selection depended only on stage one. (4) Specify the event and model, then check that the root exists, is unique, and has the required monotonicity; also verify the conditional CDF at the reported root is approximately $1/2$.