Why Selected Coefficients Are Biased

Why the Strongest Result Can Be Misleading

A search across many candidate predictors is designed to find a strong association. But even if none of the predictors has a real relationship with the outcome, random sampling variation will make some estimated slopes larger than others. Choosing the strongest estimate tends to select one that is unusually far from its true value. This selection effect is called the winner’s curse: the reported association is likely to overstate the size of the underlying association.

The issue is not that least squares produces a biased slope for a predictor chosen in advance. Rather, the search and the estimate use the same data. The selected estimate has been chosen partly because its random error happened to be favorable. The effect can be particularly striking when many predictors are searched and the largest coefficient is reported without the context of that search.

A Simple Model for Seeing the Effect

Suppose an outcome \(Y\) is modeled using a set of candidate predictors \(X_1,\ldots,X_p\):

Here \(\beta_j\) is the population slope for predictor \(X_j\), holding the other included predictors fixed. For a clean illustration, assume that the predictors are fixed and mutually orthogonal in the sample, the errors are independent with mean zero and common variance \(\sigma^2\), and the errors are normally distributed. Under these assumptions, the least-squares estimate for predictor \(j\) has a normal sampling distribution centered at its true slope. In particular, if all the candidate predictors are null, then \(\beta_j=0\) for every \(j\), and each estimate fluctuates around zero.

Orthogonality is a simplifying condition, not a requirement for selection bias. With predictor columns that are mutually orthogonal after removing the intercept (and any other regressors), and normal errors, the slope estimates are independent, which makes the selection effect especially easy to calculate. In real searches, predictors can be correlated and selection rules can be more complicated, but choosing an estimate partly because it is extreme still creates a risk of exaggeration.

A useful simplified description is \(\hat{\beta}_j\sim N(0,\tau^2)\) for every null predictor, where \(\tau\) is the common standard error. The search may select the largest positive estimate or the estimate with the largest absolute value. Those rules have an important difference: selecting the largest positive estimate produces a positive selected slope on average, whereas selecting the largest absolute estimate and keeping its original sign produces a signed average of zero under complete symmetry. In the latter case, the selected estimate is still unusually large in absolute value.

Why the Selected Estimate Is Exaggerated

The basic mechanism can be derived with just two candidate slopes. Suppose \(U\) and \(V\) are independent estimates of two null slopes, each with distribution \(N(0,\tau^2)\), and the search reports the larger estimate, \(M=\max(U,V)\), which is negative when both estimates are negative. Since \(M=(U+V+|U-V|)/2\), and both \(U\) and \(V\) have mean zero, its expectation depends on the absolute difference:

Because \(U\) and \(V\) are independent normal variables, \(U-V\sim N(0,2\tau^2)\). A mean-zero normal variable with standard deviation \(s\) has expected absolute value \(s\sqrt{2/\pi}\). Here \(s=\sqrt{2}\tau\), so \(E|U-V|=2\tau/\sqrt{\pi}\). Consequently:

The true slopes are both zero, but the larger of the two estimates has a positive expectation. This is selection bias in a direct form: the estimates themselves are centered correctly before selection, but taking the larger one changes the average of what is reported.

For a rule that selects the largest absolute estimate, write \(J\) for the selected predictor index, so that \(|\hat{\beta}_J|=\max_j|\hat{\beta}_j|\). The search has deliberately chosen an estimate with an unusually large magnitude. Under the all-null, symmetric model, a positive extreme and a negative extreme are equally likely; therefore \(E(\hat{\beta}_J)=0\) across repeated searches. But \(E|\hat{\beta}_J|\) is greater than the expected absolute value of one estimate. Once a particular search has selected, for example, a positive coefficient, its magnitude is the part most likely to be exaggerated.

Using Independent Data to Check the Magnitude

An independent sample can separate the act of choosing a predictor from the act of estimating its coefficient. Use the screening sample to select an index \(J\), then estimate the slope for that fixed selected predictor in a new sample whose outcomes were not used in the search. If the new-sample estimate is unbiased for the same population slope, then, conditional on the selected index, its expected value is the true slope:

The reason is that the independent estimate has not been chosen for being large in the new sample. If all candidate slopes are zero, this conditional expectation is zero. The validation estimate may still be positive, negative, or large by chance in a particular sample, but it does not inherit the screening sample’s winner’s-curse mechanism, provided the validation outcomes were kept separate from selection.

Independence matters. If the same outcomes are inspected repeatedly, or if the validation sample is used to change which predictor is reported, the new estimate is no longer a clean independent check. Nor does independence guarantee that the selected association will replicate: the validation estimate is subject to sampling variation. It changes the expectation and the interpretation of the estimate, not the fact that finite samples can be noisy.

Common Mistakes and Why the Distinction Matters

• Calling every selected signed coefficient upward-biased. If the rule chooses the largest absolute estimate but retains its sign, positive and negative selected coefficients balance under a completely symmetric all-null model. The average signed coefficient is zero, while the absolute magnitude is exaggerated. State which selection rule and which quantity—signed estimate or magnitude—you mean.
• Treating the selected coefficient as though the predictor had been specified in advance. Its size partly reflects the search. Interpret it in light of how many candidates and which search rules were considered.
• Assuming a small screening estimate proves the effect is absent, or a smaller validation estimate proves the first result was false. Both samples are variable. Independent data provide a less selection-affected estimate, not certainty about the population slope.
• Using validation data to search again. Choosing whichever validation result looks strongest recreates the same selection problem in the validation sample.

The practical consequence is that a selected coefficient is not a neutral summary of evidence when the same observations drove the search. The selection criterion may be useful for building a model, but the selected coefficient can overstate the association’s magnitude. Independent estimation helps make that limitation visible. The size of the winner’s curse depends on the search, the variability of estimates, and their dependence; it is not a universal correction that can be subtracted from every reported slope.