From a Discovery Signal to a Defensible Estimate
A genome-wide association signal is not yet a validated effect estimate. The discovery cohort may have been searched across many variants, and a reported locus may be included precisely because its observed association was unusually large. Independent replication can assess whether the signal persists, but a useful analysis must also check that cohorts estimate the same allele effect on the same scale. This tutorial develops those steps using a single hypothetical variant.
The analysis separates two questions. First, what effect is supported by independent replication cohorts? Second, how should the selected discovery estimate contribute once its selection event is acknowledged? These are not interchangeable: an ordinary inverse-variance average of a threshold-selected discovery estimate and replication estimates does not, by itself, remove selection bias.
Replication Planning Comes Before Looking at the Result
Replication planning specifies the effect size worth detecting, the replication sample’s anticipated standard error, the significance level, and whether the replication test is one- or two-sided. The discovery estimate is often an optimistic planning target because discovery selection favors large observed effects. A conservative assumed effect, or a range of plausible effects, is therefore preferable.
For a two-sided normal test at level \(\alpha\), if the true replication effect is \(\beta\) and its standard error is \(s\), then the test statistic has approximate distribution \(Z\sim N(\beta/s,1)\). Its power is \(P(Z>z_{1-\alpha/2})+P(Z<-z_{1-\alpha/2})\). This approximation assumes the stated standard error is appropriate and the replication cohort’s effect scale and direction are specified in advance.
Worked Example: Planning Replication Power
Suppose a team plans a two-sided replication test at \(\alpha=0.05\). For planning, it assumes a per-allele effect of \(0.050\) phenotype units and an anticipated standard error of \(0.030\) units. The standard normal critical value is approximately \(1.96\).
The noncentrality parameter is \(0.050/0.030=1.667\). Thus approximate power is \(P(Z>1.96)+P(Z<-1.96)\), where \(Z\sim N(1.667,1)\).
The planned replication has about 38.5% power under these assumptions. This is not a guarantee of failure or success; it quantifies the chance of rejecting the null in repeated studies if the assumed effect and standard error are correct. If the discovery effect was selected for being large, planning at that unadjusted value could overstate power.
Harmonize Before Combining
A meta-analysis is meaningful only when the cohort estimates refer to the same contrast. For a biallelic variant, each cohort must use the same effect allele. If cohort B reports the effect of the other allele, its coefficient changes sign: \(\hat\beta_B^{\mathrm{aligned}}=-\hat\beta_B^{\mathrm{reported}}\). The standard error is unchanged by this sign reversal. Allele frequency, strand conventions, and variant identifiers help detect mismatches; palindromic A/T or C/G variants can be ambiguous when strand cannot be resolved reliably.
The phenotype scale must also match. A per-allele linear-regression coefficient in phenotype units cannot be pooled directly with a log-odds ratio or a coefficient expressed in standard-deviation units unless a justified conversion is made. Differences in phenotype transformation, covariate adjustment, ancestry composition, imputation quality, or variant coding can create apparent heterogeneity that is not biological.
Worked Example: Aligning the Effect Allele
In a hypothetical discovery cohort, the effect allele is A and the estimate is \(0.080\) phenotype units per A allele. Replication cohort 1 also reports A as the effect allele and estimates \(0.060\) units. Cohort 2 reports the effect of the other allele, G, as \(-0.045\) units, with standard error \(0.025\).
Because the variant is biallelic, the effect per A allele in cohort 2 is the negative of its reported effect per G allele: \(\hat\beta_2^{\mathrm{aligned}}=-(-0.045)=0.045\) units. Its standard error remains \(0.025\) units. After this change, positive estimates in all three cohorts refer to the same allele contrast. If the cohort estimates had instead used different phenotype scales, this sign correction alone would not make them comparable.
Replication Meta-Analysis and Heterogeneity
For independent cohorts estimating a common effect, inverse-variance weights are \(w_j=1/s_j^2\). The fixed-effect estimate is the weighted mean, and its standard error is the square root of the reciprocal total weight. This procedure gives more weight to more precise estimates. Its common-effect interpretation is not warranted merely because a heterogeneity test is nonsignificant; substantive comparability and the uncertainty of that test matter too.
A standard measure of dispersion around the fixed-effect estimate is \(Q=\sum_jw_j(\hat\beta_j-\hat\beta_{\mathrm{FE}})^2\), compared approximately with a chi-squared distribution on \(k-1\) degrees of freedom under a common-effect model and the stated sampling assumptions. With few cohorts, this assessment has limited power to detect genuine differences. A random-effects model allows cohort effects to vary around a mean and estimates between-cohort variation; its estimate and uncertainty can differ, especially when heterogeneity is material. Model choice should be grounded in the scientific setting, not selected only by a heterogeneity p-value.
Worked Example: Pooling the Independent Replication Cohorts
Use the harmonized hypothetical replication estimates: cohort 1 has \(\hat\beta_1=0.060\) units and \(s_1=0.030\) units; cohort 2 has \(\hat\beta_2=0.045\) units and \(s_2=0.025\) units. Assume independent cohorts and a common per-allele effect for this fixed-effect calculation.
The weights are \(w_1=1/(0.030)^2=1111.1\) and \(w_2=1/(0.025)^2=1600\). The pooled estimate and standard error are
The heterogeneity statistic is approximately \(Q=1111.1(0.060-0.05115)^2+1600(0.045-0.05115)^2=0.148\). With \(df=1\), this corresponds to a large p-value, about \(0.70\). The estimates show little observed dispersion, but two cohorts provide weak information about heterogeneity. A normal-approximation 95% interval for the common effect is \(0.05115\pm1.96(0.01920)\), or approximately \(0.0135\) to \(0.0888\) phenotype units per effect allele. Under the model, this interval procedure has 95% coverage across repeated sampling; it is not a 95% probability statement about this fixed parameter.
Correcting the Selected Discovery Estimate
Suppose the discovery result was reported only because its signed statistic exceeded a prespecified threshold. In the simple one-sided selection model, the discovery statistic is \(Z=\hat\beta_D/s_D\), and selection occurs when \(Z>c\). If the true standardized effect is \(\theta=\beta/s_D\), then \(Z\sim N(\theta,1)\). Conditional on passing the threshold, its expected value is \(\theta+\lambda(c-\theta)\), where \(\lambda(a)=\phi(a)/[1-\Phi(a)]\), and \(\phi\) and \(\Phi\) are the standard normal density and distribution functions. The extra positive term describes conditional inflation among selected results.
For an observed selected statistic \(z\), the conditional likelihood is proportional to \(\phi(z-\theta)/[1-\Phi(c-\theta)]\). Its score equation sets the observed statistic equal to the model’s conditional mean: \(z=\theta+\lambda(c-\theta)\). Solving this equation yields a conditional-likelihood estimate, not a universal correction factor. It relies on knowing the selection threshold and on the normal approximation, prespecified direction, and single selection event being a reasonable description of how the result was reported.
Worked Example: Adjusting a Threshold-Selected Discovery Effect
Assume a hypothetical discovery estimate of \(0.080\) phenotype units with standard error \(0.025\), so \(z=0.080/0.025=3.20\). Suppose the reported result was selected by the one-sided rule \(Z>2.00\). Solve \(3.20=\theta+\lambda(2.00-\theta)\) numerically.
The solution is approximately \(\hat\theta=2.856\). Substitution gives \(a=2.00-2.856=-0.856\), \(\lambda(-0.856)\approx0.344\), and \(2.856+0.344\approx3.200\). The corresponding discovery effect estimate is \(0.025(2.856)=0.07140\) phenotype units per effect allele, lower than the selected estimate of \(0.080\).
This adjustment targets the mean effect under the specified truncated-normal selection model. It does not correct an incorrect allele alignment, an unknown selection process, or differences in cohort design. Nor should the original standard error automatically be treated as a complete uncertainty statement for the adjusted estimator.
Combining Selected Discovery and Replication Data
There are two defensible reporting strategies. One is to make the independent replication meta-analysis the primary estimate and report the selection-adjusted discovery result as supporting, separately described evidence. This avoids pretending that the raw discovery estimate is an ordinary independent, unselected observation. The other is to combine the discovery and replication data in a likelihood that explicitly conditions on the discovery selection event.
For the latter strategy, under a shared effect \(\beta\), independent replication meta-estimate \(r\) with standard error \(s_R\), and one-sided discovery selection \(Z_D>c\), the log likelihood, up to constants, is
The second term is essential: it accounts for the probability that a discovery statistic with effect \(\beta\) passes the threshold. Omitting it gives the usual normal likelihood and fails to account for selection. This conditional likelihood assumes the discovery selection rule is known, the replication data were not selected using the same outcome in a way that creates dependence, and the shared-effect model is appropriate. If cohorts overlap, independence is false and the joint sampling covariance must be incorporated rather than multiplying independent likelihoods.
Worked Example: Conditional-Likelihood Combination
Use the hypothetical discovery values \(d=0.080\), \(s_D=0.025\), and \(c=2.00\), together with the replication meta-estimate \(r=0.05115\) and \(s_R=0.01921\) from above. Assume independent discovery and replication samples, aligned alleles, and a common effect. Maximize the displayed conditional log likelihood over \(\beta\).
The score equation is \((z_D-\theta-\lambda(c-\theta))/s_D+(r-\beta)/s_R^2=0\), where \(\theta=\beta/s_D\). Numerical maximization gives \(\hat\beta\approx0.0558\) phenotype units per allele. At this value, \(\theta\approx2.23\), \(\lambda(2-\theta)\approx0.657\), and the discovery score contribution is about \((3.20-2.23-0.657)/0.025=12.5\). The replication contribution is about \((0.05115-0.0558)/(0.01920)^2=-12.6\); their near cancellation verifies the numerical solution to rounding.
The local observed information is approximately \(3377\) per squared phenotype unit, giving an approximate likelihood-based standard error \(1/\sqrt{3377}=0.0172\) units. A simple curvature-based interval is therefore \(0.0558\pm1.96(0.0172)\), or about \(0.0221\) to \(0.0895\) units. This interval is conditional-model based and inherits its assumptions; it should not be interpreted as repairing model misspecification or unknown genome-wide selection.
Diagnostics, Alternatives, and Limitations
Before interpreting any pooled estimate, inspect cohort-level estimates and standard errors, verify allele and scale harmonization, and check for sample overlap and discrepant phenotype or covariate definitions. Forest plots and leave-one-cohort-out analyses can reveal a sign error or an unusually influential cohort. A large \(Q\) may indicate genuine effect variation, technical incompatibility, or chance; it does not diagnose the cause. With few cohorts, a small \(Q\) does not establish homogeneity.
The main alternatives answer different questions. A fixed-effect model estimates a common effect under its assumptions. A random-effects model estimates an average across a distribution of cohort effects and incorporates between-cohort variation, but that average may not describe any particular population. A replication-only meta-analysis is transparent and protects the primary estimate from discovery selection, at the cost of discarding discovery precision. A selection-adjusted joint likelihood uses more information but depends on an explicit and often simplified model for the selection mechanism.
A common mistake is to say, “The discovery estimate has the smallest standard error, so I will include it unchanged in the inverse-variance average.” The calculation can be arithmetically correct and still be statistically misleading: selection means the discovery estimate’s conditional distribution is not its original unselected normal distribution. Another mistake is to treat a nonsignificant heterogeneity test as proof that all cohorts share the same effect. The correction is to state the model, selection rule, and independence assumptions; use harmonized data; and report sensitivity to a replication-only analysis when the selection model is uncertain.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- Why does changing the effect allele reverse a regression coefficient but leave its standard error unchanged?
- What is the fixed-effect weight for an estimate with standard error \(0.020\) phenotype units?
- Why is a nonsignificant heterogeneity test not proof that cohort effects are identical?
- What term in the conditional likelihood accounts for selection of the discovery result, and what does it represent?
- When could a replication-only meta-analysis be preferable to a joint selection-adjusted analysis?
Solutions. (1) For a biallelic variant, exchanging the counted allele changes the dosage contrast’s sign, so the coefficient changes sign; multiplication by \(-1\) does not change its standard error. (2) The weight is \(1/(0.020)^2=2500\) per squared phenotype unit. (3) A test with few cohorts can have low power to detect real effect differences, so failure to reject is not evidence that heterogeneity is absent. (4) The term \(-\log\{1-\Phi(c-\beta/s_D)\}\) conditions the discovery likelihood on passing the threshold. (5) It may be preferable when the selection mechanism is unknown or poorly represented, or when a transparent estimate based only on independent replication evidence is the desired primary result.