Median-Unbiased Estimation by Test Inversion

Why Invert Tests to Estimate After Adaptation?

When a trial stops early or selects a treatment arm based on interim results, the observed statistic does not generally have the sampling distribution it would have had under a fixed design with no selection. Inverting tests can account for that change directly: for each candidate treatment effect, use the distribution implied by the trial’s actual stopping or selection rule, then find the candidate value at which the observed result is at the center of that distribution.

The result is a median-unbiased estimate, not necessarily an unbiased estimate in the mean. It is also not automatically the same as a maximum likelihood estimate or a bias-adjusted likelihood estimate. Its justification comes from the ordering of the test statistic and the sampling distribution used in the inversion.

Definition: An estimator \(\hat{\theta}\) of a scalar treatment effect \(\theta\) is median-unbiased if, for every possible true value \(\theta\), \(P_{\theta}(\hat{\theta}\leq\theta)\geq 0.5\) and \(P_{\theta}(\hat{\theta}\geq\theta)\geq 0.5\). Equivalently, \(\theta\) is a median of the sampling distribution of \(\hat{\theta}\) under \(\theta\). If that distribution is continuous, both probabilities equal 0.5. For discrete distributions, the inequalities—not equalities—are the appropriate condition.

The distribution in this definition must correspond to the inferential target. An unconditional assessment averages over all possible adaptation paths. A conditional assessment refers to a specified event, such as having stopped early or having selected a particular arm, and uses probabilities conditional on that event. These are different properties and should not be described interchangeably.

The Inversion Principle

Let \(T\) be a statistic ordered so that larger values generally support larger values of \(\theta\). For a candidate value \(\theta_0\), a one-sided upper-tail test can be based on the tail probability \(P_{\theta_0}(T\geq t_{\mathrm{obs}})\). In an adaptive trial, this probability must be computed under the design actually used: the sampling distribution must include, or explicitly condition on, the stopping and selection rule relevant to the analysis.

For a continuous model, suppose the distribution function \(G_{\theta}(t)=P_{\theta}(T\leq t)\) is continuous and strictly decreasing in \(\theta\) at each fixed \(t\). Define the estimate \(\hat{\theta}(t)\) by the equation \(G_{\hat{\theta}(t)}(t)=0.5\), when a unique solution exists. This is the value for which the observed statistic is the median under the candidate parameter.

G_{\hat{\theta}(t)}(t)=0.5 \qquad\text{or, equivalently for a continuous distribution,}\qquad P_{\hat{\theta}(t)}(T\geq t)=0.5.

The equation is a test inversion: the one-sided test at level \(\alpha\) rejects a candidate \(\theta_0\) for sufficiently large \(T\), with rejection determined by its upper-tail probability under \(\theta_0\). The point estimate is the candidate at which that tail probability is 0.5. Thus, the procedure uses the whole family of candidate-value tests, not a test conducted only at one preselected null value.

Why the Inversion Gives a Median

The argument relies on the statistic’s ordering and the correct distribution under the design. Assume that \(T\) has a continuous distribution, that its distribution shifts monotonically upward as \(\theta\) increases, and that the equation defining \(\hat{\theta}(t)\) has a unique solution increasing in \(t\). For fixed true \(\theta\), let \(q_{\theta}\) be the median of \(T\), so that \(G_{\theta}(q_{\theta})=0.5\). By the definition of the inverted estimate, \(\hat{\theta}(q_{\theta})=\theta\). Since \(\hat{\theta}(t)\) increases with \(t\), the events \(\{\hat{\theta}(T)\leq\theta\}\) and \(\{T\leq q_{\theta}\}\) are the same. Therefore:

P_{\theta}\{\hat{\theta}(T)\leq\theta\} = P_{\theta}(T\leq q_{\theta}) = 0.5.

The corresponding upper-tail probability is also 0.5 for a continuous distribution, so \(\hat{\theta}(T)\) has median \(\theta\). This reasoning also applies to a conditional distribution if all the probabilities and quantiles in the argument are conditional on the same adaptation event. If the statistic is discrete, its distribution may jump over 0.5; an exact root may not exist, and the median inequalities must be checked rather than assuming equality.

Worked Example: Inverting a Normal Test With a Fixed Standard Error

Suppose a hypothetical trial has an estimator \(Y\) of a treatment effect \(\theta\), measured in points on a symptom scale. Assume \(Y\sim N(\theta, 4)\), so its standard error is \(2\) points, known for this example. There is no stopping or selection, and the normal model is assumed correct. If the observed value is \(y=3.0\) points, find the test-inversion median-unbiased estimate.

For a candidate \(\theta_0\), the upper-tail probability is \(P_{\theta_0}(Y\geq 3.0)=1-\Phi((3.0-\theta_0)/2)\), where \(\Phi\) is the standard normal distribution function. Set this probability to 0.5:

1-\Phi\!\left(\frac{3.0-\hat{\theta}}{2}\right)=0.5 \quad\Longrightarrow\quad \frac{3.0-\hat{\theta}}{2}=0 \quad\Longrightarrow\quad \hat{\theta}=3.0\text{ points}.

The estimate is \(3.0\) points. This equality with the observed estimator occurs because the normal model is symmetric and the standard error does not depend on \(\theta\). That convenient result should not be carried over automatically to an adaptive design: the distribution used in the inversion must then reflect the adaptation.

Conditioning on an Efficacy Stop

One way to account for an adaptation event is to conduct inference conditional on that event. The resulting estimate describes the treatment effect using the distribution of results among trials that experienced the specified event. It need not equal an unconditional estimate, which instead uses the full sampling distribution across stopping and continuation paths.

Worked Example: Conditional Inversion After Efficacy Stopping

Consider a hypothetical trial with interim statistic \(Z\sim N(\theta,1)\), where \(\theta\) is a standardized treatment effect. The trial stops for efficacy if \(Z>1.5\). Suppose it stops and the observed value is \(z=2.0\). For this example, assume the stated normal model is correct, the cutoff was fixed in advance, and the analysis is conditional on stopping. Find the conditional median-unbiased estimate.

Under a candidate value \(\theta\), the conditional distribution of \(Z\) among stopped trials has distribution function, for \(z>1.5\), \(P_{\theta}(Z\leq z\mid Z>1.5)\). The numerator is the probability of \(1.5<Z\leq z\), and the denominator is the probability of stopping. Thus:

P_{\theta}(Z\leq z\mid Z>1.5) = \frac{\Phi(z-\theta)-\Phi(1.5-\theta)} {1-\Phi(1.5-\theta)}.

Set this conditional probability at the observed \(z=2.0\) equal to 0.5. Numerically solving \(\{\Phi(2.0-\theta)-\Phi(1.5-\theta)\}/\{1-\Phi(1.5-\theta)\}=0.5\) gives \(\hat{\theta}\approx0.9275\). To check the calculation, \(1.5-0.9275=0.5725\) and \(2.0-0.9275=1.0725\); using \(\Phi(0.5725)\approx0.7165\) and \(\Phi(1.0725)\approx0.8583\), the ratio is approximately \((0.8583-0.7165)/(1-0.7165)=0.500\).

The conditional median-unbiased estimate is therefore about \(0.928\) standardized-effect units. It is below the observed \(Z\), which is plausible: among trials that crossed the efficacy boundary, the observed statistic is subject to upward truncation. The calculation is conditional on stopping; it is not a claim that this same estimate is median-unbiased unconditionally over both stopping and continuation outcomes.

Accounting for Treatment Selection

Selection can also change the distribution of the statistic even when the trial does not stop early. The next example uses a deliberately simple common-effect model to isolate that effect. It is not a model for trials in which treatment arms truly have different effects: when effects differ, selection and the target parameter require a model that represents those differences and the selection rule.

Worked Example: Correcting a Maximum for Selection

Suppose a hypothetical trial compares two treatments that, under the model used for this calculation, have the same true effect \(\theta\). Their two independent estimates satisfy \(Y_1,Y_2\sim N(\theta,1)\). The trial reports the larger estimate, \(M=\max(Y_1,Y_2)\). Assume independence and the common-effect normal model hold. If \(M=1.20\), find a median-unbiased estimate of the common effect that accounts for selecting the larger result.

The event \(M\leq m\) occurs precisely when both estimates are at most \(m\). Independence therefore gives the distribution function of the selected statistic:

P_{\theta}(M\leq m) = P_{\theta}(Y_1\leq m)P_{\theta}(Y_2\leq m) = \Phi(m-\theta)^2.

Set this probability to 0.5 at the observed \(m=1.20\). Since \(\Phi\) is increasing, \(\Phi(1.20-\hat{\theta})=\sqrt{0.5}\approx0.7071\). The standard normal quantile at 0.7071 is approximately 0.5450, so:

1.20-\hat{\theta}\approx0.5450 \quad\Longrightarrow\quad \hat{\theta}\approx0.6550.

The estimate is approximately \(0.655\) effect units, rather than the selected value \(1.20\). Under the assumed common-effect model, the median of the maximum is \(\theta+0.5450\), so subtracting this selection-related shift centers the estimator at \(\theta\) in the median sense. The correction would not justify interpreting the result as the effect of whichever arm truly has the larger effect if the two arms have different true effects.

Discrete Results, Diagnostics, and Limitations

In a discrete trial model, such as one based on a small number of binary outcomes, a tail probability changes in jumps. Consequently, solving a test-inversion equation at exactly 0.5 may have no solution. A valid median may still exist: the relevant requirement is that both \(P_{\theta}(\hat{\theta}\leq\theta)\) and \(P_{\theta}(\hat{\theta}\geq\theta)\) are at least 0.5. A procedure that reports a root as exact when the probability only jumps across 0.5 is overstating its precision. Randomized tests or a carefully specified convention for choosing among median values can address some discrete cases, but that choice must be stated.

Before using an inverted estimate, check the following features of the design and model:

Correct reference distribution. Verify that the distribution includes the actual stopping, sample-size change, or selection rule, or that the analysis consistently conditions on the specified event.
Ordering and monotonicity. Check that larger values of the statistic correspond to larger effects in the model and that the inversion has a unique, ordered solution. Nonmonotone tests can produce multiple roots or no natural point estimate.
Conditioning event. Confirm that the event has positive probability under candidate parameter values and that conditioning has not silently changed the target of inference.
Numerical stability. For a numerical root, evaluate the defining probability at the reported estimate and nearby values. The probability should be close to 0.5, subject to numerical accuracy.
Model adequacy. Dependence between interim and final statistics, nuisance parameters, and model misspecification can materially change the distribution used in the inversion.

Test inversion complements, rather than replaces, other estimation approaches. A likelihood-based estimate is obtained by optimizing a likelihood; a bias-adjusted likelihood estimate modifies that objective or its score. Stagewise ordering estimates use an ordering of outcomes across stages to construct an estimate. Test inversion instead asks which candidate parameter makes the observed ordered result central under the specified test family. Different methods can yield different estimates because they use different orderings, conditioning choices, and targets.

The estimate also does not, by itself, give an uncertainty interval. Nor does the term “median-unbiased” mean that the estimate is unbiased in expectation, that every realized estimate is close to the true effect, or that the probability the parameter is below the estimate is 0.5 after observing the data. The guarantee concerns repeated sampling under the stated model and design. Its practical value depends on whether those assumptions represent the trial adequately.

Common pitfall: Treating a selected or stopped statistic as though it came from the original fixed-design distribution can ignore the very event that changed its sampling behavior. Instead, derive the distribution under the actual adaptation rule, state whether inference is conditional or unconditional, and invert tests based on that distribution.

Specify the target and adaptation.
Name the treatment effect and the stopping or selection rule relevant to the analysis.

Construct the test distribution.
For each candidate effect, use the full design distribution or a clearly specified conditional distribution.

Invert at the median level.
Find the candidate value giving a central tail probability of 0.5; assess existence and uniqueness.

Verify the interpretation.
Check the median property under the relevant repeated-sampling distribution and report the conditioning and units.

Check Your Understanding

Answer each question using the stated design and inferential target.

For a continuous estimator distribution, what equation defines a median-unbiased estimate through the lower-tail distribution function?
Why can the estimate in the efficacy-stopping example be lower than the observed statistic?
In the two-treatment example, what assumption allows the distribution of the maximum to be written as a product of two normal distribution functions?
Why is an exact 0.5 tail-probability solution not always available for a discrete statistic?
What must be clarified before calling an estimate after adaptation conditionally median-unbiased?

Solutions. (1) The inversion solves \(P_{\hat{\theta}}(T\leq t_{\mathrm{obs}})=0.5\), under a continuous model and the correct sampling distribution. (2) Conditioning on crossing the efficacy boundary truncates the distribution; among stopped trials, values are necessarily above the cutoff, so the observed statistic can be high relative to the conditional distribution’s center. (3) Independence of the two estimates, together with their common-effect model, makes \(P(Y_1\leq m,Y_2\leq m)=P(Y_1\leq m)P(Y_2\leq m)\). (4) Discrete tail probabilities jump over 0.5, so equality may be impossible even though a median can satisfy both inequalities. (5) Specify the conditioning event, use the sampling distribution conditional on that event, and state the model and design assumptions supporting it.