Deriving a Plug-In Correction for Selection Bias

Why Correct the Selected Estimate?

When several treatment arms are compared at an interim analysis, the arm that proceeds is often the one with the most promising observed estimate. Even if each estimate is unbiased before selection, the estimate for an arm selected because it looked favorable tends, on average, to overstate that arm’s true effect. A plug-in correction makes this conditional bias explicit: estimate the expected selection error under a fitted model, then subtract it from the observed estimate.

The correction is conditional on a specified selection event. It is not a universal adjustment for every estimate called “selected,” and it does not turn the selected estimate into an exactly unbiased estimate by definition. Its quality depends on whether the selection rule and the model used to calculate the expected error are appropriate.

The symbol \(b_j\) describes an expectation over repeated trials under the model, conditional on the arm being selected. It is not the error in a particular observed estimate. The plug-in step replaces unknown parameters in that expectation with estimates; it does not observe the conditional expectation directly.

Deriving the Correction for a Two-Arm Normal Selection

Consider two treatment-effect estimates \(X_1\) and \(X_2\), with independent sampling errors and a common known standard error \(\sigma\): $$ X_1\sim N(\theta_1,\sigma^2),\qquad X_2\sim N(\theta_2,\sigma^2). $$ Suppose arm 1 is selected when \(X_1>X_2\), and define \(\delta=\theta_1-\theta_2\). The target is the conditional bias \(E(X_1-\theta_1\mid X_1>X_2)\). The normal model, independence, equal standard errors, and this exact selection rule are assumptions of the derivation.

Set \(D=X_1-X_2\). Then \(D\sim N(\delta,2\sigma^2)\), and \(\operatorname{Cov}(X_1-\theta_1,D)=\sigma^2\). Because the pair is jointly normal, the conditional mean of \(X_1-\theta_1\) given \(D\) is its linear regression on \(D\): $$ E(X_1-\theta_1\mid D)=\frac{\sigma^2}{2\sigma^2}(D-\delta)=\frac{1}{2}(D-\delta). $$ Taking the expectation over \(D>0\) gives the conditional selection bias. To evaluate it, standardize \(D\): \(Z=(D-\delta)/(\sqrt{2}\sigma)\). The event \(D>0\) is \(Z>z\), where \(z=-\delta/(\sqrt{2}\sigma)\). For a standard normal variable, \(E(Z\mid Z>z)=\phi(z)/[1-\Phi(z)]\). Substitution yields $$ b_1(\delta)=\frac{\sigma}{\sqrt{2}}\, \frac{\phi(z)}{1-\Phi(z)},\qquad z=-\frac{\delta}{\sqrt{2}\sigma}. $$

This bias is positive: conditional on winning, \(X_1\) tends to include a positive sampling error. Since \(\delta\) is generally unknown, the plug-in version replaces it with an estimate \(\widehat\delta\), while retaining the model’s assumed \(\sigma\). A good estimate of \(\delta\) matters: when the fitted model says arm 1 was unlikely to win on its true effect, the observed win may be attributed to a larger positive sampling error.

Plugging In Parameters More Generally

The same construction applies beyond this two-arm example. Let \(\boldsymbol{\eta}\) collect the parameters needed to describe the joint distribution of estimates and let \(S\) specify the full selection event relevant to the reported estimate. If the selected estimate is \(X_J\), define its conditional bias as \(b(\boldsymbol{\eta})=E_{\boldsymbol{\eta}}(X_J-\theta_J\mid S)\). Given a fitted value \(\widehat{\boldsymbol{\eta}}\), evaluate that conditional expectation under the fitted model and subtract it from the observed selected estimate.

Two choices must be made carefully. First, \(S\) must describe how the estimate came to be reported. If selection depended on several interim comparisons or stopping decisions, conditioning only on the final arm’s identity may omit relevant information. Second, \(\widehat{\boldsymbol{\eta}}\) must be obtained in a way that supports the model calculation. The conditional expectation can depend on the effects of competing arms, standard errors, and covariance between estimates, as well as on the target arm’s effect.

The plug-in correction is often attractive because it is direct: it uses a model-based conditional expectation rather than requiring that a separate unbiased estimate of the selected effect be available. But estimating the nuisance parameters can itself be difficult after selection. In particular, reusing an unusually favorable selected estimate to fit the parameters that predict its own selection error may understate or otherwise distort the correction.

Assumptions, Diagnostics, and Alternatives

A plug-in correction is most defensible when the following conditions are reasonably supported:

• The selection event is represented correctly. Include the comparisons, thresholds, and stages that determine whether the estimate is observed or reported.
• The sampling model is adequate. The joint distribution, standard errors, and any dependence between arm estimates should reflect the design. Shared-control estimates, for example, are generally not independent.
• The fitted parameters are credible. The bias can be sensitive to estimated effects and standard errors. A parameter estimate drawn from the selected result itself may be especially vulnerable to selection.
• The target is stated clearly. Identify whether the corrected quantity concerns the effect for the selected arm, a particular selection path, or another estimand. These are not interchangeable targets.

Useful checks include comparing the fitted model’s predicted selection frequency with the design’s observed or expected frequency, checking that simulated covariance and standard errors match the design, and examining how the correction changes under plausible alternative parameter values. In a simulation implementation, too few retained trials make the estimated conditional bias noisy; rare selection events may require substantially more simulations. A correction with an unexpected sign or an implausibly large size should prompt a check of the selection-event coding, the contrast’s sign convention, and the fitted parameters.

The unadjusted estimate is simpler but does not account for conditional upward error. A plug-in correction directly subtracts the model’s predicted conditional bias, but its reliability depends on the fitted model and it does not, by itself, provide a selection-adjusted confidence interval. Conditional maximum likelihood is an alternative likelihood-based approach that conditions the likelihood on selection; it estimates parameters by a different procedure and does not make plug-in bias subtraction unnecessary in every setting. The choice should follow the estimand, the available information, and the plausibility of the selection model.

Common Mistake and Key Limitation

A frequent mistake is to subtract the unconditional bias, \(E(X_j-\theta_j)\), from an estimate selected by a rule. An estimator can have zero unconditional bias while having substantial conditional bias among selected trials. The correction must correspond to the event that actually determined selection.

Another mistake is to describe \(X_{j,\mathrm{corr}}\) as “unbiased” merely because a fitted conditional bias was subtracted. The true conditional bias is evaluated at unknown parameters; plugging in estimates introduces additional error, and model misspecification can add more. The corrected estimate is a model-based adjustment, not a guarantee. Its uncertainty must account for the estimation and selection process separately if interval estimation is required.