Winner’s Curse After Selecting Associated Variants

Why Selection Makes Genetic Effects Look Larger

A genetic variant is often carried forward because its estimated association with an exposure or mediator is statistically strong. That rule is understandable: variants with stronger observed associations may seem more informative. But the estimate used to select a variant contains sampling error. Among estimates that cross a threshold, those with positive noise are more likely to be selected than those with negative noise. The selected estimates therefore tend to be farther from zero than their underlying effects.

This selection-induced exaggeration is called the winner’s curse. It can affect genetic mediation when variants are selected for strong exposure associations or mediator associations and the selected estimates are then used in an indirect-effect calculation. The size and direction of the distortion depend on what was selected and how the estimates enter the analysis; it is not correct to say that every downstream causal estimate must be inflated.

Definition: Suppose an estimate \(\hat\theta\) is selected when it meets an association criterion. Winner’s curse is the difference between its conditional expectation after selection and the underlying effect: \(E(\hat\theta\mid S)-\theta\), where \(S\) denotes the selection event. For a two-sided threshold and a fixed effect coding, the conditional bias can be positive for a positive underlying effect, even when \(\hat\theta\) is unbiased before selection.

The distinction between an unbiased estimate before selection and a biased estimate among selected results is central. Before selection, random errors above and below the true effect balance on average. After selecting only estimates with sufficiently large absolute test statistics, that balance no longer holds within the selected set.

Conditional Bias Under a Two-Sided Threshold

Consider a single genetic effect \(\theta\) estimated by \(\hat\theta\), with a known standard error \(SE\). Assume the sampling model

\hat\theta\sim N(\theta,SE^2).

Assume the estimate is selected if its two-sided test statistic exceeds a fixed threshold \(c\):

S=\left\{\left|\frac{\hat\theta}{SE}\right|>c\right\}.

This calculation assumes the normal model is appropriate, the standard error is treated as fixed, the threshold is fixed, and the effect is expressed using a fixed allele coding. It describes selection of this estimate by its own association statistic. It does not by itself account for dependence between estimates, selection of a maximum from many variants, or additional selection steps.

Deriving the Selection Bias

Standardize the estimate by writing \(Z=\hat\theta/SE\) and \(\delta=\theta/SE\). Then \(Z\sim N(\delta,1)\), and selection occurs when \(Z>c\) or \(Z<-c\). The probability of selection is

P(S)=1-\Phi(c-\delta)+\Phi(-c-\delta),

where \(\Phi\) is the standard normal cumulative distribution function. For a normal variable \(Z\sim N(\delta,1)\), the contribution to \(E(Z-\delta)\) from the upper selected tail is \(\phi(c-\delta)\), and the contribution from the lower selected tail is \(-\phi(-c-\delta)\), where \(\phi\) is the standard normal density. Dividing the sum of these contributions by the probability of selection gives the conditional bias:

E(\hat\theta-\theta\mid S) = SE\frac{\phi(c-\delta)-\phi(-c-\delta)} {1-\Phi(c-\delta)+\Phi(-c-\delta)}.

Because the normal density is symmetric, \(\phi(-c-\delta)=\phi(c+\delta)\). For \(\theta>0\), the upper-tail contribution will generally outweigh the lower-tail contribution, making the conditional signed estimate larger than \(\theta\). Under a zero effect, the two tails balance and the conditional mean of the signed estimate is still zero. However, the selected estimates’ absolute magnitudes are large by construction. If effect signs are chosen from the observed estimates, the signed-effect formula above is not the formula for the resulting sign-aligned magnitudes.

Worked Example: Bias for One Selected Effect

Suppose a hypothetical variant has true effect \(\theta=0.10\) outcome units per allele, and its estimate has \(SE=0.05\) outcome units per allele. It is selected when \(|\hat\theta/SE|>1.96\). Calculate the expected signed estimate among selected results.

Here \(\delta=0.10/0.05=2\), \(c-\delta=-0.04\), and \(c+\delta=3.96\). The selection probability is

P(S)=1-\Phi(-0.04)+\Phi(-3.96)\approx 0.5160.

Using \(\phi(-0.04)\approx0.3986\) and \(\phi(3.96)\approx0.0001570\), the conditional bias is

E(\hat\theta-\theta\mid S) = 0.05\frac{0.3986-0.0001570}{0.5160} \approx 0.0386 \quad\text{outcome units per allele}.

Thus the expected selected estimate is \(0.10+0.0386=0.1386\) outcome units per allele. The estimate is unbiased before selection under the stated model, but its conditional expectation after this selection rule is about 38.6% above the true effect.

How Selection Enters an Indirect-Effect Product

In a simple mediation calculation, let \(a\) denote one component effect and \(b\) the other, with indirect effect \(\psi=ab\). The exact meaning of the components depends on the mediation design; for example, the product method uses an exposure-to-mediator effect and a mediator-to-outcome effect. If estimated components are selected for strong associations, their conditional expectations can differ from their underlying values.

When the two estimates are independent and the selection events are made separately using each estimate, their conditional product expectation factorizes:

E(\hat a\hat b\mid S_a,S_b) = E(\hat a\mid S_a)E(\hat b\mid S_b).

Consequently, selection can distort the indirect-effect product even if each component estimate was unbiased before selection. This factorization requires independence of the component estimates and their selection events. With correlated errors, including some forms of sample overlap, the conditional product also depends on their conditional covariance; multiplying the two conditional means is then not enough.

Worked Example: Selection Bias in an Indirect-Effect Product

Consider a hypothetical mediation analysis with true component effects \(a=0.20\) mediator units per allele and \(b=0.15\) outcome units per mediator unit. The true indirect effect is \(ab=0.0300\) outcome units per allele. Assume independent normal component estimates, each selected using a two-sided threshold of 1.96.

For \(\hat a\), take \(SE=0.10\), so \(\delta_a=0.20/0.10=2\). The selection probability is approximately 0.5160. Using the corresponding densities from the previous calculation, its conditional bias is

0.10\frac{0.3986-0.0001570}{0.5160} \approx 0.0772 \quad\text{mediator units per allele}.

Therefore \(E(\hat a\mid S_a)\approx0.20+0.0772=0.2772\) mediator units per allele.

For \(\hat b\), take \(SE=0.10\), so \(\delta_b=0.15/0.10=1.5\). Here \(c-\delta_b=0.46\) and \(c+\delta_b=3.46\). The selection probability and conditional bias are approximately

P(S_b)=1-\Phi(0.46)+\Phi(-3.46)\approx0.3230,

E(\hat b-b\mid S_b) = 0.10\frac{\phi(0.46)-\phi(3.46)}{0.3230} \approx 0.10\frac{0.3589-0.001003}{0.3230} \approx0.1108 \quad\text{outcome units per mediator unit}.

Thus \(E(\hat b\mid S_b)\approx0.2608\) outcome units per mediator unit. Under the stated independence assumption, the expected selected product is

E(\hat a\hat b\mid S_a,S_b) \approx0.2772(0.2608)=0.0723 \quad\text{outcome units per allele}.

This exceeds the true product \(0.0300\) outcome units per allele. It is an illustration of conditional component inflation, not a general prediction that every mediation estimate will have this size or direction of bias.

Not Every Downstream Estimate Is Inflated

The effect of selection depends on the estimator. In the product calculation above, positive component estimates are multiplied, so upward shifts in both conditional means can produce an upward shift in the product. But a ratio has a denominator: if a selected exposure association is used as the denominator of a ratio estimate, making that denominator larger can pull the ratio downward. Selection affecting a numerator can act differently. Moreover, uncertainty, covariance, and selection on multiple related quantities may all matter.

The practical lesson is to trace selection through the actual formula rather than label the entire analysis “inflated.” Identify which estimate triggered selection, whether the selected quantity enters as a numerator, denominator, or product component, and whether the relevant errors are independent. In mediation, also keep the statistical selection issue separate from the causal assumptions needed to interpret an indirect effect.

Replication and Shrinkage

An independent replication study estimates the same effect using new data, without selecting the variant again based on the replication estimate. Under the normal model, a replication estimate has expectation \(\theta\) before any replication-based filtering. It will vary around the true effect; it is not guaranteed to be closer to \(\theta\) in any one study. If variants are retained only when their replication estimates also pass a threshold, selection bias can arise again.

Shrinkage offers a different response. A simple normal-prior model makes the idea explicit: assume \(\theta\sim N(0,\tau^2)\) and observe an estimate \(y\mid\theta\sim N(\theta,SE^2)\). The posterior mean is a weighted average of the estimate and prior mean zero. It pulls noisy estimates toward zero, with more shrinkage when the sampling variance is large relative to the prior variance. The method relies on the prior being a reasonable model for the effects under consideration; a poorly chosen prior can mislead.

Worked Example: Shrinking a Selected Estimate

Suppose a hypothetical variant has an observed estimate \(y=0.30\) outcome units per allele with \(SE=0.10\), and it crossed a two-sided threshold of 1.96 in the discovery data. For illustration, assume the prior model \(\theta\sim N(0,0.20^2)\). Under this model, the posterior mean is

E(\theta\mid y) = \frac{\tau^2}{\tau^2+SE^2}y = \frac{0.04}{0.04+0.01}(0.30) = 0.240 \quad\text{outcome units per allele}.

The posterior variance is \(\tau^2SE^2/(\tau^2+SE^2)=0.008\), so the posterior standard deviation is \(\sqrt{0.008}\approx0.0894\) outcome units per allele. A normal-model 95% posterior interval is approximately \(0.240\pm1.96(0.0894)\), or \([0.0647,0.415]\) outcome units per allele.

This calculation illustrates shrinkage under the stated prior, not a universal correction factor. Given the exact observed estimate \(y\), the fact that it passed a deterministic threshold adds no further information to the posterior; the threshold is already determined by \(y\). If only the selection event or a coarsened summary were observed, the likelihood would need to account for truncation. In practice, shrinkage should reflect a defensible effect distribution and the information actually available.

Common Mistakes and Practical Checks

Calling the original estimate biased before selection. Under the stated sampling model, \(E(\hat\theta)=\theta\). The bias described here is conditional on the selection event.
Assuming every selected signed estimate is upwardly biased. For a two-sided threshold, the signed conditional mean under a true zero effect remains zero. Sign-aligned magnitudes are a different quantity.
Assuming all causal estimates become larger. Product and ratio estimators respond differently to which component was selected. Trace the selection through the estimator’s formula.
Multiplying conditional component means when estimates are dependent. The product factorization in the example requires independence. With dependence, the conditional covariance must also be considered.
Treating replication or shrinkage as automatic cures. Replication can be selected again, and shrinkage depends on its model. Neither removes the need to describe the selection process.

Key takeaway: Association-based variant selection changes the distribution of the estimates carried forward. Quantify that change for the selected component, then trace it through the mediation estimator while stating the assumptions about dependence, selection, and any shrinkage model.

Check Your Understanding

Use the assumptions stated in each question, and distinguish unconditional from selection-conditional quantities.

Why can an estimate be unbiased before selection yet have a positive conditional bias after a two-sided association threshold?
In the normal-threshold model, what does \(\delta=\theta/SE\) represent, and how does it enter the selection probability?
For an indirect-effect estimate \(\hat a\hat b\), when can its conditional expectation be written as the product of the conditional component means?
Why does selection that inflates an exposure association not necessarily inflate a ratio estimate using that association as its denominator?
What additional selection problem can arise if a variant is retained only when its independent replication estimate also passes a significance threshold?