Bias-Adjusted Maximum Likelihood Estimates

Why adjust an estimate after treatment selection?

When an interim comparison selects the treatment arm with the larger observed effect, the selected arm’s interim estimate is likely to be high relative to its true effect. This selection effect can remain in the final estimate even when additional independent data are collected after selection. A bias adjustment aims to quantify and subtract the expected contribution of that interim selection.

The adjustment considered here is deliberately specific: a normal two-arm model with known, equal variances, selection of the arm with the larger stage-one estimate, and an equal-precision stage-two estimate for the selected arm. Under these assumptions, the correction has a simple derivation. Outside them, its formula need not apply.

Definition: If $S_1$ is the event that arm 1 is selected, the conditional bias of an estimator $\hat{\theta}_1$ for $\theta_1$, given that arm 1 was selected, is $E(\hat{\theta}_1-\theta_1\mid S_1)$. A bias-adjusted estimate subtracts an estimate of this conditional bias from the ordinary estimate. Subtracting a plug-in estimate of bias does not, by itself, make the resulting estimator exactly unbiased.

Model and ordinary maximum likelihood estimate

Let $\theta_1$ and $\theta_2$ denote the fixed treatment effects for two candidate arms, measured relative to a common reference. At stage one, let $X_1$ and $X_2$ be the estimated effects for the two arms. Assume

X_i\sim N(\theta_i,\sigma^2),\qquad i=1,2,

with $X_1$ and $X_2$ independent and with known $\sigma>0$. Arm 1 is selected when $X_1>X_2$; write this event as $S_1$. Ties have probability zero in this continuous model. Following selection of arm 1, an independent stage-two estimate $Z_1$ is observed, with $Z_1\sim N(\theta_1,\sigma^2)$. The stage-two estimate is independent of both stage-one estimates, conditional on the fixed effects.

For the selected arm, the ordinary likelihood for its two independent estimates is proportional to $\exp\{ -[(X_1-\theta_1)^2+(Z_1-\theta_1)^2]/(2\sigma^2)\}$. Maximizing it over $\theta_1$ gives the equal-weight pooled estimate

\hat{\theta}_{1,\mathrm{MLE}}=\frac{X_1+Z_1}{2}.

This is the ordinary maximum likelihood estimate under the stated model. It does not account for the fact that $X_1$ had to exceed $X_2$ for arm 1 to be selected. The stage-two estimate is independent of that selection event and has conditional mean $\theta_1$; the stage-one contribution does not.

Deriving the selection-bias adjustment

Set $\delta=\theta_1-\theta_2$, the true difference between the candidate treatment effects, and define $D=X_1-X_2$. Then $D\sim N(\delta,2\sigma^2)$, and selection of arm 1 is the event $D>0$. The stage-one estimate $X_1$ and contrast $D$ are jointly normal, with

\operatorname{Cov}(X_1-\theta_1,D-\delta)=\sigma^2,\qquad \operatorname{Var}(D)=2\sigma^2.

For jointly normal variables, the conditional mean of one centered variable given the other is its covariance-to-variance ratio times the observed centered value. Thus $X_1-\theta_1=\frac12(D-\delta)+R$, where $R$ is mean zero and independent of $D$. Consequently, conditioning on selection gives

E(X_1-\theta_1\mid S_1)=\frac12 E(D-\delta\mid D>0).

To calculate the truncated-normal expectation, put $a=\delta/(\sqrt{2}\sigma)$ and $W=(D-\delta)/(\sqrt{2}\sigma)$. Then $W\sim N(0,1)$, selection is $W>-a$, and $P(S_1)=\Phi(a)$, where $\Phi$ is the standard normal cumulative distribution function and $\phi$ is its density. Since $\int_{-a}^{\infty}w\phi(w)\,dw=\phi(a)$,

E(W\mid W>-a)=\frac{\phi(a)}{\Phi(a)},\qquad E(D-\delta\mid D>0)=\sqrt{2}\sigma\frac{\phi(a)}{\Phi(a)}.

Combining these expressions yields the conditional bias in the stage-one estimate. The pooled maximum likelihood estimate gives stage one half the total weight, while $Z_1$ contributes zero conditional bias. Therefore the pooled estimate’s bias is half the stage-one bias.

Result: Under the specified normal, equal-variance model, the conditional bias of the pooled estimate when arm 1 is selected is $$ B_1(\delta)=E(\hat{\theta}_{1,\mathrm{MLE}}-\theta_1\mid S_1) =\frac{\sigma}{2\sqrt{2}}\frac{\phi(a)}{\Phi(a)},\qquad a=\frac{\delta}{\sqrt{2}\sigma}. $$ This bias is positive for every finite $\delta$. It decreases as arm 1 becomes more likely to be selected because it truly has the larger effect.

Worked calculations

Worked Example: Equal true effects

Suppose $\theta_1=\theta_2$ and $\sigma=1$ effect unit. Then $\delta=0$ and $a=0$. By symmetry, each arm is selected with probability $0.5$. The stage-one conditional bias is

E(X_1-\theta_1\mid S_1) =\frac{1}{\sqrt{2}}\frac{\phi(0)}{\Phi(0)} =\frac{1}{\sqrt{2}}\frac{0.3989}{0.5000} \approx 0.5642\ \text{effect units}.

The pooled estimate gives stage one half weight, so its conditional bias is $B_1(0)=0.5642/2\approx0.2821$ effect units. This is a conditional expectation among trials selecting arm 1, not a claim that every selected estimate is inflated by exactly this amount.

Worked Example: A nonzero true difference

Suppose the true contrast is $\delta=0.80$ effect units and $\sigma=1$ effect unit. Then $a=0.80/\sqrt{2}\approx0.5657$, with $\Phi(a)\approx0.7142$ and $\phi(a)\approx0.3400$. The selection probability for arm 1 is about 0.7142. Its pooled-estimate bias, conditional on selection, is

B_1(0.80) =\frac{1}{2\sqrt{2}}\frac{0.3400}{0.7142} \approx 0.1683\ \text{effect units}.

The expected inflation is smaller than in the equal-effects example: when arm 1 truly has the larger effect, selection is less often driven by a favorable random fluctuation. The adjustment remains an expectation under the model, not an observed-data guarantee.

Worked Example: Plug-in adjustment from hypothetical trial data

Suppose a hypothetical trial has $\sigma=1$ effect unit and observes $X_1=0.50$, $X_2=0.10$, and $Z_1=0.30$ effect units. Since $0.50>0.10$, arm 1 is selected. The ordinary pooled estimate is

\hat{\theta}_{1,\mathrm{MLE}}=\frac{0.50+0.30}{2}=0.40\ \text{effect units}.

For a simple plug-in correction, estimate the true contrast $\delta$ by the stage-one contrast $D=X_1-X_2=0.40$. This gives $\hat{a}=0.40/\sqrt{2}\approx0.2828$, $\Phi(\hat{a})\approx0.6114$, and $\phi(\hat{a})\approx0.3833$. The estimated bias and adjusted estimate are

\widehat{B}_1 =\frac{1}{2\sqrt{2}}\frac{0.3833}{0.6114} \approx0.2217\ \text{effect units},\qquad \hat{\theta}_{1,\mathrm{adj}} =0.40-0.2217 =0.1783\ \text{effect units}.

This is a plug-in estimate, because the unknown contrast in the model-based bias formula has been replaced by observed data. The adjusted value is not a new observation, and it is not guaranteed to equal the true effect or to be closer to it in this particular trial.

What the adjustment does—and does not do

The calculation separates two ideas that are easy to conflate. The ordinary maximum likelihood estimate combines the two independent estimates for the selected arm. The bias-adjusted estimate takes that ordinary estimate and subtracts an estimate of its conditional mean inflation under the specified selection rule. It is not the same as maximizing a likelihood that explicitly conditions on selection. Conditional likelihood estimation is a distinct approach; the earlier material on conditional likelihood estimation after selection develops that construction.

The adjustment also does not provide a confidence interval or establish its coverage. Subtracting an estimated bias changes the point estimate, but uncertainty remains about the treatment effect, the selection event, and the estimated correction itself. Interval construction and coverage require separate methods.

Estimate	Construction	Interpretation
Ordinary MLE	Pool $X_1$ and $Z_1$ with equal weights	Maximum likelihood estimate under the unadjusted normal sampling model
Plug-in bias-adjusted estimate	Subtract the fitted conditional-bias formula from the ordinary MLE	A correction for estimated mean selection inflation under the stated model
Conditional likelihood estimate	Use a likelihood that accounts for conditioning on the selection event	A different estimation procedure, not equivalent in general to subtracting the plug-in correction

Assumptions, diagnostics, and common pitfalls

Before using this formula, check that its assumptions match the design and analysis. The stage-one estimates must be approximately normal with the same known variance, and they must be independent. The selection rule must be exactly “choose the larger stage-one estimate.” The stage-two estimate must be independent of stage one and have the same variance, so that the ordinary pooled estimate gives the two observations equal weight. The formula also presumes that the target is the selected arm’s fixed treatment effect relative to the same reference.

In application, inspect the design records and analysis definitions rather than relying only on a normal approximation. Verify the actual selection rule, whether variances and sample sizes differ across arms, whether the stage-two estimator uses independent data, and whether the effect scale is consistent across stages. A shared control, unequal precision, additional selection criteria, or a data-dependent sample size can change the covariance structure or the weighting. In such cases, this derivation cannot simply be reused by substituting a new standard error.

A frequent pitfall is to treat the plug-in correction as an exact removal of bias. The formula $B_1(\delta)$ is the conditional bias when the true contrast $\delta$ is known. In practice, the contrast is unknown, and replacing it with $D$ introduces estimation variability. The adjusted estimate therefore need not have zero conditional bias. A second pitfall is to subtract the stage-one bias from the whole pooled estimate: because stage one has weight one half, the appropriate pooled correction in this model is $B_1(\delta)$, the formula derived for the pooled estimate, not the larger stage-one bias $\frac{\sigma}{\sqrt{2}}\frac{\phi(a)}{\Phi(a)}$.

Wrong reasoning: “The model gives the bias exactly, so subtracting its plug-in value makes the final estimate unbiased.”
Correction: The model gives an exact conditional expectation only when its parameters are known and its assumptions hold. A plug-in correction estimates that expectation; it does not guarantee zero bias for the adjusted estimator.

Finally, an adjustment can be unstable when the selection model is poorly matched to the design or when the estimated contrast is noisy. The direction and size of the correction should be reported alongside the assumptions and the ordinary estimate. A large difference between the two estimates is a reason to check the model and selection process carefully, not proof that one estimate is correct.

Check Your Understanding

Use the equal-variance model in this tutorial. Solutions are included after the questions.

When $\delta=0$, what is the probability that arm 1 is selected, and what are the stage-one and pooled conditional biases for $\sigma=1$?
For $\delta=0.80$ and $\sigma=1$, calculate $a$ and state whether the pooled conditional bias is larger or smaller than when $\delta=0$.
Suppose the stage-one bias is $0.50$ effect units and the pooled estimate gives stage one half weight. What is the pooled conditional bias under this setup?
In the hypothetical data $X_1=0.30$, $X_2=0.10$, and $Z_1=0.20$, with $\sigma=1$, calculate the ordinary pooled estimate. What additional calculation is needed for the plug-in adjustment?
Why does subtracting the plug-in bias estimate not guarantee an unbiased estimate?

Solutions. (1) By symmetry $P(S_1)=0.5000$. The stage-one bias is $0.5642$ effect units; the pooled bias is half of that, $0.2821$ effect units. (2) $a=0.80/\sqrt{2}\approx0.5657$. The pooled bias is smaller than at $\delta=0$, approximately $0.1683$ rather than $0.2821$ effect units. (3) The pooled bias is $0.50/2=0.25$ effect units. (4) The ordinary estimate is $(0.30+0.20)/2=0.25$ effect units. To calculate the plug-in adjustment, first use $D=X_1-X_2=0.20$ to form $\hat{a}=D/(\sqrt{2}\sigma)$, then evaluate the bias formula and subtract it. (5) The plug-in correction uses an estimated contrast rather than the unknown true contrast, and its validity depends on the model assumptions; the estimated correction need not equal the actual conditional bias.

Estimate	Construction	Interpretation
Ordinary MLE	Pool \(X_1\) and \(Z_1\) with equal weights	Maximum likelihood estimate under the unadjusted normal sampling model
Plug-in bias-adjusted estimate	Subtract the fitted conditional-bias formula from the ordinary MLE	A correction for estimated mean selection inflation under the stated model
Conditional likelihood estimate	Use a likelihood that accounts for conditioning on the selection event	A different estimation procedure, not equivalent in general to subtracting the plug-in correction