Power Calculations for Replication of Selected Loci

Why Replication Power Needs a Selection-Aware Interpretation

A replication study is planned for loci selected because their discovery results passed a threshold. Its power depends on the effect that will apply in the replication population, the replication sample size, allele frequency, and the test threshold. A discovery estimate is not automatically a suitable substitute for that effect: among selected loci, estimated effects tend to be larger than their underlying effects. This selection effect, often called the winner’s curse, can make a calculation based on the discovery estimate substantially optimistic.

The calculation below separates two questions. First, what is the probability that the prespecified replication test will reject its null hypothesis if the true effect is a specified value? Second, what would that probability be if an inflated discovery estimate were incorrectly treated as the true effect? The second calculation can illustrate the size of the bias, but it should not replace the first.

Model and Power Formula

Definition: Let $G$ be the number of copies of a specified allele, coded $0,1,2$, and let $f$ be its frequency. Under Hardy–Weinberg equilibrium, $\operatorname{Var}(G)=2f(1-f)$. In a quantitative-trait linear regression with residual standard deviation $\sigma$, an additive effect $\beta$ is the expected change in the trait per additional allele, in the trait’s units. With $N$ independent participants and no material loss of information from covariate adjustment, an approximate standard error is $$ SE(\hat{\beta}) \approx \frac{\sigma}{\sqrt{N\,2f(1-f)}}. $$

This approximation assumes the same effect and residual variance apply to the replication cohort, independent participants, correctly measured genotypes, an additive model, and a sufficiently accurate normal approximation to the estimator. If the phenotype is standardized, $\sigma=1$ and $\beta$ is in standard-deviation units. Otherwise, $\beta$ and $SE$ retain the phenotype’s units. The formula is most directly useful for a quantitative trait; binary outcomes, related participants, substantial covariate information loss, or complex sampling can require a model-specific information calculation or simulation.

For a prespecified two-sided replication test at level $\alpha$, let $c=z_{1-\alpha/2}$, the corresponding standard-normal critical value. Under a true positive effect, the test statistic is approximately $Z\sim N(\delta,1)$, where $\delta=\beta/SE$. The rejection region is $Z>c$ or $Z<-c$. Therefore,

\operatorname{Power}(\beta) = P(Z>c\mid\beta)+P(Z<-c\mid\beta) = 1-\Phi(c-\delta)+\Phi(-c-\delta).

Here $\Phi$ is the standard normal cumulative distribution function. For a negative effect, the same two-sided power results after replacing $\delta$ by its absolute value. The second tail is often very small when the effect is in the expected direction, but it belongs in the formula. The replication threshold is set by the replication testing plan; it need not equal the discovery threshold.

Worked Example: Power at a Specified True Effect

Suppose a hypothetical independent replication cohort has $N=2{,}500$ participants. The allele frequency is $f=0.20$, the quantitative trait has residual standard deviation $\sigma=1$, and the true additive effect is $\beta=0.10$ trait standard deviations per allele. The prespecified replication test is two-sided at $\alpha=0.05$. The model assumptions above are taken to hold for this illustration.

The genotype variance is $2(0.20)(0.80)=0.32$, so $$ SE \approx \frac{1}{\sqrt{2{,}500(0.32)}}=\frac{1}{\sqrt{800}}=0.03536 $$ trait standard deviations per allele. Thus the noncentrality parameter is $$ \delta=\frac{0.10}{0.03536}=2.828. $$ For $\alpha=0.05$, $c=1.960$. Substitution gives $$ \operatorname{Power} =1-\Phi(1.960-2.828)+\Phi(-1.960-2.828) =\Phi(0.868)+\Phi(-4.788) \approx 0.807. $$

The planned test has approximately 80.7% power under this model if the true replication effect is 0.10 standard deviations per allele. This is not the probability that the effect is true or that the study will replicate; it is the rejection probability conditional on the stated effect and assumptions.

How Selection Inflates a Discovery Estimate

To make the selection effect precise, suppose the discovery estimator has known standard error $SE_D$, and under a common true effect $\beta$, $$ Z_D=\frac{\hat{\beta}_D}{SE_D}\sim N(\delta_D,1), \qquad \delta_D=\frac{\beta}{SE_D}. $$ For a locus selected because its discovery statistic is in the positive tail, selection can be represented by $Z_D>c_D$. Conditional on this event, the expected $Z_D$ exceeds its unselected mean $\delta_D$. The result follows directly by integrating the normal density.

E[Z_D\mid Z_D>c_D] =\frac{\int_{c_D}^{\infty}z\phi(z-\delta_D)\,dz} {1-\Phi(c_D-\delta_D)} =\delta_D+\frac{\phi(c_D-\delta_D)} {1-\Phi(c_D-\delta_D)}.

For the integral, set $u=z-\delta_D$, so $z=u+\delta_D$. The integral of $\delta_D\phi(u)$ over the selected tail is $\delta_D[1-\Phi(c_D-\delta_D)]$. Since $\phi'(u)=-u\phi(u)$, the remaining integral of $u\phi(u)$ is $\phi(c_D-\delta_D)$. Dividing by the tail probability yields the expression above. The added term is positive, so the conditional expected discovery estimate $SE_D E[Z_D\mid Z_D>c_D]$ exceeds $\beta=SE_D\delta_D$.

This is a conditional mean over repeated discovery studies under the specified model, not a correction that exactly recovers the true effect for every selected locus. Selection rules based on two tails, multiple correlated variants, or additional filters can change the conditional distribution. In practice, the selection process should be represented as accurately as possible if estimating or correcting selection bias is the goal.

Worked Example: Comparing True-Effect and Plug-In Power

Consider a hypothetical locus with true effect $\beta=0.08$ trait standard deviations per allele. Suppose $SE_D=0.020$, so $\delta_D=0.08/0.020=4$. For illustration, assume a locus is selected in the positive direction when $Z_D>5$. Its conditional mean discovery statistic is $$ E[Z_D\mid Z_D>5] =4+\frac{\phi(1)}{1-\Phi(1)} =4+\frac{0.24197}{0.15866} \approx 5.525. $$ The corresponding expected discovery estimate is $0.020(5.525)=0.1105$ standard deviations per allele, rather than the true value $0.08$.

Now suppose the independent replication cohort has $SE_R=0.040$, and uses a two-sided $\alpha=0.05$ test. Under the true effect, $\delta_R=0.08/0.040=2.000$, so $$ \operatorname{Power}_{\text{true}} =1-\Phi(1.960-2.000)+\Phi(-1.960-2.000) =\Phi(0.040)+\Phi(-3.960) \approx 0.516. $$ If the expected, selection-inflated discovery estimate were wrongly substituted as the true effect, the noncentrality parameter would instead be $0.1105/0.040=2.763$. The resulting plug-in power is $$ \operatorname{Power}_{\text{plug-in}} =1-\Phi(1.960-2.763)+\Phi(-1.960-2.763) =\Phi(0.803)+\Phi(-4.723) \approx 0.789. $$

The large difference is not evidence that selection changes the replication cohort’s sampling distribution. Under the assumed common true effect and independent cohorts, replication power conditional on that effect remains about 51.6%, even after conditioning on a discovery selection event. The plug-in calculation is optimistic because it uses an estimate whose expected value was increased by the selection rule.

Allele Frequency and Replication Sample Size

The standard-error formula shows why allele frequency matters even when the per-allele effect is fixed. Under the additive model, $2f(1-f)$ is largest at $f=0.50$, and smaller for less common alleles. Holding $N$, $\sigma$, and $\beta$ fixed, lower genotype variance means a larger standard error, a smaller noncentrality parameter, and typically lower power. This comparison assumes the same effect scale and a valid additive model at each frequency; it does not imply that effects are biologically equal across populations.

Worked Example: Comparing Two Allele Frequencies

Suppose two hypothetical replication settings each have $N=7{,}000$, a standardized quantitative trait ($\sigma=1$), and true effect $\beta=0.08$ standard deviations per allele. Use a two-sided $\alpha=0.05$ test, and assume the model and sampling conditions hold in both settings.

At $f=0.10$, $2f(1-f)=0.18$, giving $$ SE=\frac{1}{\sqrt{7{,}000(0.18)}}=\frac{1}{\sqrt{1{,}260}}=0.02817, \qquad \delta=\frac{0.08}{0.02817}=2.839. $$ The power is $1-\Phi(1.960-2.839)+\Phi(-1.960-2.839)$, or $\Phi(0.879)+\Phi(-4.799)\approx0.811$.

At $f=0.50$, $2f(1-f)=0.50$, so $$ SE=\frac{1}{\sqrt{7{,}000(0.50)}}=\frac{1}{\sqrt{3{,}500}}=0.01690, \qquad \delta=\frac{0.08}{0.01690}=4.733. $$ The power is $\Phi(4.733-1.960)+\Phi(-4.733-1.960)\approx0.997$. Under these assumptions, the commoner allele yields greater power because it provides more genotype variation in this sample, not because its per-allele effect is larger.

Planning, Diagnostics, and Alternatives

A useful planning workflow makes the assumptions visible rather than hiding them in a single power number.

Define the replication target.
Specify the variant, phenotype, effect scale, test threshold, and the effect value for which power is being calculated. Use the estimand intended for replication, not automatically the discovery point estimate.

Calculate information.
Use allele frequency and planned sample size to obtain an approximate standard error. For the quantitative-trait model here, check that the genotype coding, residual standard deviation, and sample-size assumptions match the setting.

Compute power and vary assumptions.
Evaluate the two-sided formula at the planned $\alpha$, then recalculate across plausible effects, allele frequencies, sample sizes, and selection-aware effect scenarios.

Diagnostics begin with the inputs. Confirm whether the frequency refers to the same allele in both cohorts; verify that the effect is per allele and on the stated trait scale; and check that the residual standard deviation and sample size refer to the analyzable replication sample. Compare the assumed and observed genotype variance when available. If the replication cohort has different ancestry, phenotype measurement, or covariate structure, the discovery effect and standard-error approximation may not transfer.

A normal approximation is a warning sign when expected minor-allele counts are small, outcome data are sparse, or estimates are strongly affected by separation or relatedness. For binary outcomes, the variance and information depend on case-control allocation and the fitted model; replacing them with the quantitative-trait formula without justification can misstate power. A model-based calculation or simulation that reproduces the intended analysis is preferable in such cases.

The main alternatives answer different needs. A direct plug-in calculation using the observed discovery estimate is simple but can overstate power after selection. A conservative planning calculation can use a smaller, externally justified effect, though that choice should be stated rather than presented as a selection correction. Simulating the discovery selection and independent replication stages can represent complicated selection rules and estimator behavior, but requires a credible data-generating model. These methods do not eliminate uncertainty about the true effect; they make different assumptions about it.

Common Pitfall and Limits

Common mistake: “The selected locus had a large discovery estimate, so I will use that estimate as the true effect and report the resulting replication power.”

Correction: Power is conditional on a specified true effect and a replication design. Selection makes the discovery estimate’s conditional distribution biased upward in magnitude, so its plug-in power can be optimistic. Report power at a clearly identified effect value, explain its source, and show sensitivity to smaller plausible effects. When discovery and replication effects may differ, a common-effect calculation alone is not sufficient.

No power calculation guarantees replication. A null result can occur even when power is high, and a nominally significant replication can occur by chance under the null. The calculation also does not establish that a selected association is causal, resolve population differences, or account automatically for all uncertainty in the effect estimate and model inputs. Its value is as a transparent description of expected rejection probability under stated assumptions.

Check Your Understanding

Answer each question using the assumptions stated in the question. Brief solutions are included for checking.

For a quantitative trait with $\sigma=1$, $N=1{,}600$, and $f=0.25$, calculate the approximate standard error per allele. If the true effect is $0.10$, what is $\delta$?
Why can a selected discovery estimate produce larger plug-in replication power than power calculated at the true effect?
At fixed $N$, $\sigma$, and per-allele effect, why does $f=0.10$ generally give lower power than $f=0.50$ under the additive model?
If replication is independent of discovery conditional on the true effect, does conditioning on discovery significance itself change replication power? Explain.

Solutions. (1) The genotype variance is $2(0.25)(0.75)=0.375$, so $SE=1/\sqrt{1{,}600(0.375)}=1/\sqrt{600}=0.04082$; hence $\delta=0.10/0.04082=2.449$. (2) Selection favors unusually large discovery statistics, so the selected estimate is expected to exceed the true effect in magnitude; substituting it increases the calculated noncentrality parameter. (3) The genotype variance is $0.18$ at $f=0.10$, compared with $0.50$ at $f=0.50$; lower genotype variance increases $SE$ and reduces $|\delta|$. (4) No. Under the stated independence and common-effect assumptions, replication’s rejection probability conditional on the true effect and design is unchanged; selection matters when interpreting the discovery estimate used to guess that effect.