Why Sensitivity Analysis Is Part of Estimation
A GWAS heritability estimate depends not only on the phenotype and sample, but also on analysis choices. The included SNPs determine which genetic variation is represented; an LD reference approximates the correlation structure used by the estimator; ancestry covariates can affect control of population structure; and a binary trait may be reported on an observed or liability scale. A single estimate does not show how much these choices matter.
A sensitivity analysis makes those choices explicit and changes them in a planned way. Its purpose is not to find the largest estimate, the smallest standard error, or the specification that agrees with a preferred conclusion. It is to show how the estimate and its uncertainty respond to defensible alternatives, and to identify which differences reflect a changed target rather than instability in estimating one fixed target.
For a GWAS summary-statistic analysis using LD score regression (LDSC), suppose the reported quantity is an estimate of SNP heritability, written here as \(\hat h^2_{\mathrm{SNP}}\), with standard error \(SE\). The estimate concerns the additive genetic variance tagged by the included variants under the model and scale used. In particular, changing the SNP set can change what genetic variation is represented. Such a comparison is not necessarily a comparison of two noisy estimates of exactly the same quantity.
A useful baseline states the choices in concrete terms: which GWAS and phenotype definition are used, which variants enter, which LD scores are used, which ancestry covariates are included, and whether the result is on the observed or liability scale. Alternatives should be scientifically defensible, and one should change one major choice at a time before considering combinations. This makes the source of a difference easier to locate.
Worked Example: A One-Choice-at-a-Time Comparison
Worked Example: A One-Choice-at-a-Time Comparison
Consider a hypothetical case-control GWAS analysed with LDSC. Every value below is illustrative. The baseline uses a stated common-variant SNP set, an ancestry-matched LD reference, ancestry covariates, and the observed scale. The other rows change one choice at a time; all other choices and the GWAS are held fixed.
| Specification | \(\hat h^2_{\mathrm{SNP}}\) | \(SE\) |
|---|---|---|
| Baseline | 0.0128 | 0.0018 |
| More restricted SNP set | 0.0108 | 0.0017 |
| Expanded SNP set | 0.0139 | 0.0021 |
| Alternative LD reference | 0.0116 | 0.0019 |
| Ancestry covariates omitted | 0.0157 | 0.0023 |
For the baseline, an approximate 95% Wald confidence interval is the estimate plus or minus \(1.96\) standard errors:
Under the baseline model and its usual large-sample approximation, the interval describes uncertainty in the observed-scale estimate. It does not incorporate uncertainty due to choosing a different SNP set or LD reference. The restricted-SNP estimate is \(0.0020\) below baseline, while the expanded-SNP estimate is \(0.0011\) above it. These shifts are useful to report, but the SNP-set alternatives may represent different tagged-variant targets, not merely alternative calculations of an identical target.
Using the alternative LD reference changes the estimate by \(0.0116-0.0128=-0.0012\). This is evidence that the result is sensitive to the LD input in this analysis; it is not evidence that the reference producing the larger estimate is preferable. Omitting ancestry covariates raises the estimate by \(0.0029\). That change warrants investigation, but it does not by itself establish that the baseline or the unadjusted result is unbiased. Each standard error describes the estimator under its own specification and should be displayed alongside its estimate.
Comparing Differences and Their Uncertainty
The standard errors in a sensitivity table are not, by themselves, the standard error of the difference between two estimates. Specifications applied to the same GWAS use overlapping data, so their estimates are generally correlated. Treating the estimates as independent and calculating a difference standard error as \(\sqrt{SE_1^2+SE_2^2}\) discards that covariance and can give a misleading comparison.
A paired resampling approach can preserve the dependence. For example, divide the genome into \(B\) approximately independent blocks and refit both specifications after deleting each block. For each deletion, calculate the difference between the two estimates. The delete-block differences, rather than two separate sets of standard errors, then quantify uncertainty in the contrast.
Worked Example: A Paired Block-Jackknife Contrast
Suppose five hypothetical delete-block fits give differences between the estimate without ancestry covariates and the baseline estimate of \(0.0022,\ 0.0031,\ 0.0027,\ 0.0038,\) and \(0.0027\). Their mean is
For delete-one-block estimates, the jackknife standard error of the contrast is
Here the deviations from \(0.0029\) are \(-0.0007,\ 0.0002,\ -0.0002,\ 0.0009,\) and \(-0.0002\). Therefore,
This calculation preserves the pairing of the two specifications within each block deletion. With only five blocks, the uncertainty estimate would be too coarse to treat as a stable inferential result; a real analysis would use a suitable, substantially larger set of genomic blocks. The example shows why uncertainty for a difference needs a method that accounts for dependence, not just the marginal standard errors in a results table.
Changing the Trait Scale
For a binary trait, observed-scale heritability depends on the case fraction in the analysed sample. A liability-scale estimate instead expresses heritability under a threshold model in which an underlying continuous liability gives rise to case status. The conversion requires the population prevalence \(K\) and the case fraction \(P\) in the analysed sample. It also assumes the threshold-liability model and that these quantities are treated as known for the conversion.
Worked Example: Converting an Estimate to the Liability Scale
Continue with the hypothetical baseline estimate \(0.0128\) and standard error \(0.0018\). Suppose the assumed population prevalence is \(K=0.10\), and cases make up \(P=0.20\) of the analysed sample. These are hypothetical inputs, not estimates from a real study.
The threshold is \(t=\Phi^{-1}(0.90)\approx1.2816\), and the standard normal density there is \(\phi(t)\approx0.1755\). Substitution gives
The liability-scale estimate and standard error are therefore
A correspondingly transformed approximate 95% interval is \(0.0210\pm1.96(0.00296)\), or approximately \((0.0152,\ 0.0268)\). This uncertainty calculation treats the prevalence inputs as fixed. If \(K\) or \(P\) is uncertain, that uncertainty is not represented by simply multiplying the original standard error by \(C\); it requires additional propagation or sensitivity analysis. Different plausible prevalence assumptions can therefore change both the converted estimate and its uncertainty.
A Practical Analysis Sequence
A clear sensitivity analysis is easier to interpret when the order of decisions is explicit. The following sequence separates the estimate, its standard error, and the target to which it refers.
Record the phenotype, GWAS sample, SNP set, LD reference, ancestry adjustment, trait scale, estimate, and standard error.
For each analysis choice, state why an alternative is reasonable. Vary one choice at a time first so its effect is identifiable.
Give each estimate and standard error, and use intervals where appropriate. State whether a change modifies the target, the model input, or the scale.
Do not infer the uncertainty of a difference from marginal standard errors alone. Use paired resampling or another method that accounts for dependence when formal contrast uncertainty is needed.
Describe which choices have the largest effects and whether the conclusion depends on a particular specification. Do not select a preferred estimate merely because it is larger or more precise.
Common Mistake: Treating Sensitivity as a Search for the Best Number
A common mistake is to inspect several specifications, select the largest heritability estimate or the one with the smallest standard error, and present it as the best estimate. This is not justified: the larger value may reflect a changed SNP target, an LD-reference mismatch, or residual population structure, while the smaller standard error measures precision under that particular model rather than its validity.
The correction is to state the baseline and the alternatives before interpreting results, report the estimate and uncertainty for each, and explain what each change means for the target and assumptions. Agreement across defensible specifications can support robustness to those particular choices; disagreement identifies sensitivity that should be reported and investigated. Neither pattern alone settles every question about the interpretation of SNP heritability.
Check Your Understanding
Answer each question by referring to the distinctions between target, estimate, and uncertainty.
- Why can changing the SNP set alter the quantity being estimated rather than only the precision of the estimate?
- In the hypothetical comparison, what is the absolute change from the baseline when ancestry covariates are omitted? Why does that difference alone not identify which estimate is unbiased?
- Why is \(\sqrt{SE_1^2+SE_2^2}\) generally not an appropriate standard error for the difference between two estimates from the same GWAS?
- Which assumptions and inputs are needed to convert an observed-scale binary-trait estimate to the liability scale?
- If the prevalence inputs are treated as fixed, how does multiplying the observed-scale estimate by a positive conversion factor change its standard error and confidence interval?