Winner’s Curse and Replication Effect Sizes

Why Discovery Estimates Can Be Too Large

A genetic association study does not usually report every estimated effect. It emphasizes variants whose test statistics pass a stringent threshold. Even if an effect estimate is unbiased before selection, estimates that pass a threshold tend to be unusually large in the selected direction: some reflect the true association, and some reflect sampling error that happened to push the estimate farther from zero. This selection-induced exaggeration is called the winner’s curse.

The name does not imply that a reported association is necessarily false. It describes a property of the estimated effect after selection. A real association can be discovered correctly while its initial estimated effect is still too large on average. This matters when planning replication studies, interpreting effect sizes, and using discovery estimates to predict power or clinical impact.

Definition: The winner’s curse is the tendency for effect estimates selected because they are sufficiently large, often because they pass a significance threshold, to overstate the underlying effect in the selected direction. The effect estimate may be unbiased before selection; the exaggeration arises from conditioning on the selection event.

To make the mechanism precise, consider a single genetic effect on a specified phenotype scale. Let \(\beta\) be the true per-allele effect, in the units of that scale, and let \(\hat{\beta}_D\) be its estimate in a discovery sample. Suppose the estimate has a known standard error \(SE_D\) and, under a normal approximation, \(\hat{\beta}_D\sim N(\beta,SE_D^2)\). For the derivations below, assume the standard error is treated as fixed, the model and phenotype scale are correctly specified, and the discovery estimate is approximately normally distributed.

Model: Before selection, \(\hat{\beta}_D=\beta+SE_D Z\), where \(Z\sim N(0,1)\). For a one-sided discovery rule in a specified positive direction, a result is selected when \(\hat{\beta}_D/SE_D>c\), where \(c\) is the prespecified threshold on the standardized estimate.

The one-sided rule is a simplifying model for the mathematics. Many association analyses use a two-sided significance threshold and then describe the estimated effect in its observed direction. The same basic mechanism applies to selected magnitudes: among estimates that pass a two-sided threshold, sampling error has helped make the magnitude large. The exact conditional formula changes with the selection rule, so an adjustment must match the rule that was actually used.

How Selection Changes the Expected Estimate

Under the one-sided model, selection is equivalent to requiring \(Z>a\), where \(a=c-\beta/SE_D\). A standard normal variable truncated below at \(a\) has conditional mean \(\phi(a)/[1-\Phi(a)]\), where \(\phi\) and \(\Phi\) are the standard normal density and cumulative distribution functions. Substituting into the model gives the expected discovery estimate among selected results.

E(\hat{\beta}_D\mid \hat{\beta}_D/SE_D>c) = \beta+SE_D\frac{\phi(c-\beta/SE_D)}{1-\Phi(c-\beta/SE_D)}

The fraction is positive, so under this model the conditional mean of selected estimates exceeds \(\beta\). The amount of expected exaggeration depends on both the selection threshold and the signal-to-noise ratio \(\beta/SE_D\). When the true effect is small relative to its standard error, passing the threshold requires a particularly favorable sampling error, so the selection effect can be substantial. When the true effect is large relative to its standard error, selection is more likely even without unusually favorable error, and the conditional exaggeration is generally smaller.

Worked Example: Expected Exaggeration After a Threshold

Suppose, hypothetically, that a variant’s true effect is \(0.040\) phenotype units per allele and its discovery standard error is \(0.020\) units per allele. The analysis selects estimates with \(Z>1.96\). Under the normal model, calculate the mean estimate among selected results.

Here, \(\beta/SE_D=0.040/0.020=2.00\), so the truncation point for the standard normal error is \(a=1.96-2.00=-0.040\). Thus \(\phi(a)\approx0.3986\) and \(1-\Phi(a)=\Phi(0.040)\approx0.5160\). The conditional mean is

0.040+0.020\left(\frac{0.3986}{0.5160}\right) = 0.040+0.020(0.7725) \approx 0.05545

The true effect is \(0.040\) units per allele, while the mean estimate among results passing this one-sided threshold is about \(0.0555\) units per allele. The difference, about \(0.0155\) units per allele, is the model-based expected selection exaggeration for this setup. It is an average over selected samples, not a guarantee that any particular discovery estimate is exaggerated by that amount.

Discovery and Replication Are Not Contradictory

A replication estimate is not expected to reproduce the selected discovery estimate exactly. If the replication sample is independent of discovery, uses a comparable phenotype and effect scale, and estimates the same underlying effect, its estimate is not conditioned on the discovery threshold. Under those assumptions, \(E(\hat{\beta}_R\mid\text{discovery selection})=\beta\), where \(\hat{\beta}_R\) is the replication estimate. The discovery estimate, in contrast, is conditionally shifted upward under the selection model.

This difference is a form of regression to the mean: an extreme selected estimate is likely to be less extreme in a new independent sample, even when the association is real. Regression to the mean is not, by itself, evidence against the association. Nor does a smaller replication estimate prove that the original result was false. The replication estimate also has sampling uncertainty, and its precision depends on its standard error and sample size.

The comparison is informative only when the estimates are meaningfully comparable. Differences in phenotype definition, effect scale, allele coding, or population context can produce differences that are not explained by winner’s curse alone. Allele harmonization and phenotype alignment are therefore essential when interpreting replication effects. Independence matters too: overlapping participants can make discovery and replication estimates correlated, so the independent-sample expectation above would no longer describe their comparison.

Worked Example: A Smaller Replication Estimate

Consider hypothetical, harmonized estimates for the same allele and phenotype. The discovery estimate is \(0.061\) units per allele with \(SE_D=0.015\); an independent replication estimate is \(0.034\) units per allele with \(SE_R=0.020\). The discovery result passed its selection rule, while the replication estimate was not selected on the basis of its own size.

The estimates differ by \(0.061-0.034=0.027\) units per allele. That difference does not show that the discovery association was false. Selection makes a high discovery estimate more likely to be unusually large, while independent replication targets the underlying effect without conditioning on the discovery estimate’s size. The replication estimate is also uncertain: its standard error is \(0.020\) units per allele, so the point estimate alone does not establish that the true effect is exactly \(0.034\).

A careful interpretation is that the replication estimate is smaller and more compatible with regression toward the underlying effect than exact repetition of the discovery estimate would be. Whether the combined evidence supports an association requires assessing the estimates and their uncertainty, not treating their difference as a contradiction. This interpretation assumes comparable measurements and independent samples; it does not explain away every possible discrepancy.

A Simple Selection-Adjusted Estimate

A direct way to illustrate correction under the one-sided model is to invert the conditional-mean equation. If an observed estimate has standardized value \(z_{\mathrm{obs}}\), seek a value \(\delta=\beta/SE_D\) for which the expected standardized estimate among selected results equals \(z_{\mathrm{obs}}\). The equation is

z_{\mathrm{obs}} = \delta+ \frac{\phi(c-\delta)}{1-\Phi(c-\delta)}, \qquad \hat{\beta}_{\mathrm{adj}}=SE_D\delta

This conditional-mean inversion is a simple model-based shrinkage adjustment: the adjusted estimate is pulled back from the observed discovery estimate to account for the expected upward shift caused by selection. In general, the equation is solved numerically. It is not the only possible correction, and it should not be interpreted as an exact unbiased estimate for an individual result. Its value is pedagogical as well as practical: it makes the selection rule and the assumed sampling model explicit.

Worked Example: Inverting the Selection Effect

Suppose a hypothetical discovery estimate is \(0.060\) units per allele, with \(SE_D=0.020\) units per allele, and the one-sided selection threshold is \(c=1.96\). Its observed standardized estimate is \(z_{\mathrm{obs}}=0.060/0.020=3.00\). Solve the conditional-mean equation for \(\delta\).

Try \(\delta=2.52\). Then \(c-\delta=1.96-2.52=-0.56\), with \(\phi(-0.56)\approx0.3413\) and \(1-\Phi(-0.56)=\Phi(0.56)\approx0.7123\). Therefore,

2.52+\frac{0.3413}{0.7123} \approx 2.52+0.4793 \approx 2.9993

This is approximately the observed standardized value of \(3.00\). The adjusted effect estimate is therefore

\hat{\beta}_{\mathrm{adj}} = 0.020(2.52) = 0.0504 \text{ units per allele}

The simple adjustment reduces the discovery estimate from \(0.060\) to about \(0.0504\) units per allele. The result depends on the assumed normal sampling model and the one-sided threshold. It is not a new observation from an independent sample, and it does not remove uncertainty about the true effect.

Assumptions, Limits, and Common Pitfalls

The conditional calculation is only as credible as its inputs. It assumes that the reported standard error is suitable for the sampling distribution, that the effect estimate is approximately normal, and that the selection event has been described correctly. In a real analysis, selection may involve a genome-wide threshold, multiple phenotype or subgroup searches, correlated variants, or other reporting decisions. A correction based only on one threshold can understate the full selection process if those additional choices also influenced which result was reported.

The method also treats the target effect as comparable between discovery and replication. Genuine differences in allele frequency, linkage disequilibrium, phenotype measurement, or population-specific effects can complicate that assumption. A lower replication estimate may reflect some combination of selection, sampling variability, and real differences between settings. The simple model isolates one mechanism; it does not diagnose every cause of replication differences.

A common mistake is to reason, “The replication estimate is smaller, so the discovery association must have been spurious.” That conclusion does not follow. Selection predicts that discovery estimates passing a threshold will be larger on average than independent estimates, even when the true effect is nonzero. The correction is not to dismiss the discovery result, but to report the estimates with their uncertainty, consider the selection process, and interpret replication evidence in context. Conversely, winner’s curse should not be used as a blanket explanation for any disagreement: substantial inconsistency still deserves investigation.

Key takeaway: Significance-based selection changes the distribution of reported effect estimates. Compare discovery and replication effects with that conditioning in mind, and label any shrinkage or correction with its model and selection assumptions.

Check Your Understanding

Answer each question using the assumptions stated in the question.

In the one-sided normal model, why is the expected estimate among selected results larger than the true effect?
A discovery estimate is significant and an independent, comparable replication estimate is smaller. Give one explanation consistent with a real association, and state one assumption needed for it.
For the conditional-mean adjustment, what roles do the selection threshold and standard error play?
Why is a selection-adjusted estimate not the same thing as an independent replication estimate?
Name two reasons, other than winner’s curse, why discovery and replication effect estimates might differ.