Why Simulate a Sequencing Design?
Power calculations for rare-variant studies can depend strongly on assumptions that are hard to compress into a single formula: which variants are present, how their frequencies differ by ancestry, how effects vary in direction and size, and how often sequencing misses or miscalls a genotype. A simulation makes those assumptions explicit. It lets a design team ask what its planned analysis would do across plausible scenarios, rather than relying on one idealized genetic architecture.
A simulation is not a guarantee that a future study will have a particular result. It is a conditional calculation: if the specified population, recruitment, sequencing, phenotype, and analysis assumptions were true, how often would the analysis reject the null? That rejection proportion estimates power under an alternative. Repeating the same process under a null scenario estimates false-positive risk.
Specify the Study Before Simulating
For a binary phenotype, one useful starting point is a source population with ancestry group \(K\), covariates \(C\), and true genotypes \(G_1,\ldots,G_m\) at the \(m\) variants in a gene or region. A possible phenotype model is
Here \(Y=1\) denotes a case, \(\gamma_K\) represents ancestry-associated differences in baseline risk, and \(\beta_j\) is the effect of variant \(j\). Under a gene-level null scenario, set all gene effects to zero; under an alternative, specify which variants are causal and how their effects are distributed. The equation is a simulation choice, not a claim that every disease follows a logistic model.
The genotype distribution should reflect ancestry-specific allele frequencies and, where material, linkage disequilibrium. Genotypes can be generated from haplotypes or other joint distributions; independently drawing every variant is a simplifying assumption that should be stated, not silently imposed. The sequencing step then turns true genotypes into observed calls, allowing errors and missingness to depend on genotype, ancestry, batch, or case status if that is plausible.
Finally, reproduce the intended recruitment and analysis. In a cohort design, simulate the cohort and observe its cases and controls. In a case-control design, recruit the planned numbers of cases and controls from the source population, or simulate genotype distributions conditional on case status. Apply the same quality filters, ancestry adjustment, gene-level test, and significance threshold intended for the study. If the actual analysis uses a test calibrated for small samples, simulate that exact procedure rather than substituting a convenient approximation.
The scenario grid should vary the factors most likely to change the conclusion. These commonly include total sample size and case-control allocation; ancestry proportions and ancestry-specific allele frequencies; disease prevalence and recruitment scheme; sequencing sensitivity, false-call rates, and missingness; and genetic architecture, including the number of causal variants, effect sizes, and effect directions. Change one factor at a time to understand its contribution, then evaluate combinations that are jointly plausible.
From Simulated Studies to Power
For each scenario, generate \(R\) independent simulated studies and apply the prespecified analysis to each. Let \(I_r=1\) if replicate \(r\) rejects \(H_0\), and \(I_r=0\) otherwise. The estimated power under an alternative is the mean of these indicators. Under a null scenario, the same calculation estimates the false-positive rate.
Because each \(I_r\) is a Bernoulli outcome with rejection probability \(\pi\), its variance is \(\pi(1-\pi)\). Independence across simulation replicates gives
This Monte Carlo standard error measures simulation noise in the estimated rejection proportion. It does not measure uncertainty about the true allele frequencies, effect sizes, or error rates used as inputs. A larger \(R\) reduces Monte Carlo noise, but does not make unrealistic assumptions more credible.
Worked Example: Comparing Two Sample Sizes
Suppose a hypothetical gene-based study has 1,000 cases and 1,000 controls in its smaller design, with a fixed analysis and significance threshold. Under a specified alternative, 640 of 2,000 simulated studies reject \(H_0\). A larger design has 1,500 cases and 1,500 controls; it rejects in 1,090 of 2,000 simulated studies. Assume the same ancestry composition, phenotype model, sequencing process, and genetic architecture in both scenarios.
For the smaller design,
Its Monte Carlo standard error is
For the larger design, estimated power is \(1090/2000=0.545\), with Monte Carlo standard error \(\sqrt{0.545(0.455)/2000}\approx 0.0111\). Under these hypothetical assumptions, increasing the sample size raises estimated power from 32.0% to 54.5%. The Monte Carlo errors are about 1.0 and 1.1 percentage points, respectively; they are much smaller than the estimated difference, but the result still depends on the assumed effect sizes and other inputs.
Null Scenarios and False-Positive Risk
A design should be evaluated under null scenarios as well as alternatives. Setting every gene effect to zero is necessary, but may not be sufficient. If ancestry predicts phenotype risk and allele frequencies differ by ancestry, an unadjusted analysis can produce a gene-level signal even though the gene has no effect within ancestry groups. Similar concerns arise if case status is associated with sequencing batch and batch affects call quality.
To assess these risks, simulate the relevant structure under the gene-level null and run the complete planned analysis. Include the intended ancestry adjustment, matching or stratification, quality filters, and any test-selection steps. If several tests or genes will be examined, reproduce the planned multiplicity correction or selection procedure. A nominal threshold for one test does not automatically control the chance of at least one false positive across many tests.
Worked Example: Checking Ancestry Confounding
Consider a hypothetical null gene whose variants have different frequencies across two ancestry groups. Disease risk also differs between the groups, but the gene has no effect within either group. In 2,000 simulations with ancestry proportions balanced between cases and controls, an unadjusted test rejects in 101 replicates. In a separate scenario with ancestry imbalance between cases and controls, it rejects in 286 replicates. When the planned ancestry adjustment is applied to the imbalanced scenario, it rejects in 108 replicates.
The estimated false-positive rates are, respectively,
For the imbalanced, unadjusted scenario, the Monte Carlo standard error is \(\sqrt{0.143(1-0.143)/2000}\approx 0.00783\). The rate of 14.3% is therefore far above the intended nominal 5% level, not merely a small fluctuation around it. In this hypothetical setting, ancestry adjustment brings the estimated rate closer to 5%. That result supports evaluating the adjustment in the design simulation; it does not establish that adjustment will eliminate confounding under every possible ancestry structure.
Sequencing Error and Genetic Architecture
A useful simulation distinguishes true genotypes from observed calls. For example, a simple error model can specify \(P(G^{\mathrm{obs}}=g'\mid G=g)\), the probability that a true genotype \(g\) is recorded as \(g'\). A more detailed model may generate read depth and alternate-allele counts before calling genotypes. Either way, error parameters should reflect the planned assay and quality-control process. If errors differ between cases and controls or between batches, simulate that difference: nondifferential error can reduce power, while differential error can also create false-positive signals.
Genetic architecture matters for the same reason. A burden-style analysis can lose power when causal effects point in opposing directions, while a variance-component or adaptive procedure may behave differently. Use a small set of clearly defined architectures rather than one convenient effect pattern: for example, all causal variants increasing risk, mixed directions, or a smaller fraction of causal variants. For each, state the number of causal variants and how their frequencies and effect sizes are assigned. The analysis method and its calibration must be fixed or its full selection process must be represented in every replicate.
Worked Example: Mixed Effects and Imperfect Calls
Suppose a hypothetical study uses one prespecified, calibrated gene-level procedure at the same threshold in all scenarios. Under an alternative with effects in a common direction and perfect calls, 712 of 1,000 simulated studies reject. Under an alternative with effects in mixed directions and perfect calls, 548 reject. Under that same mixed-direction architecture with a specified sequencing-error model, 431 reject.
The estimated powers are
For the mixed-direction scenario with error, the Monte Carlo standard error is \(\sqrt{0.431(0.569)/1000}\approx 0.0157\). In these hypothetical simulations, power is lower with mixed effects than with common-direction effects, and lower again when the specified call errors are introduced. These are properties of this chosen architecture, assay model, and analysis together; they are not universal correction factors that can be applied to another study.
A Practical Simulation Workflow
A disciplined workflow helps make comparisons interpretable and repeatable.
Specify the population, ancestry groups, recruitment scheme, case-control allocation, phenotype prevalence where relevant, and planned sample size.
Set gene effects to zero for null scenarios. For alternatives, define the causal fraction, effect sizes, directions, and ancestry-specific frequencies or effects to be explored.
Simulate genotypes, phenotypes, recruitment, and sequencing errors in the order implied by the study. Preserve relevant ancestry, linkage, batch, and ascertainment features.
Use the intended quality filters, covariates, gene-level test, calibration, and significance threshold on every replicate.
Report rejection proportions, Monte Carlo standard errors, and the scenario assumptions. Add plausible adverse scenarios, especially for confounding and error.
Common Pitfalls and Reporting
- Simulating only the favorable alternative. High power is not reassuring if the null analysis is inflated. Include null scenarios with plausible ancestry imbalance and technical differences.
- Changing the analysis after seeing simulated results. Repeatedly choosing the best-performing test on the same simulations can make performance look optimistic. Prespecify the test, or simulate the complete test-selection procedure and account for it in the threshold.
- Treating Monte Carlo precision as scientific certainty. A small Monte Carlo standard error only means the simulated rejection proportion is estimated precisely for the inputs supplied. It does not validate those inputs.
- Varying several assumptions without recording them. Keep a scenario table with sample size, ancestry proportions, prevalence or sampling scheme, error model, architecture, analysis, threshold, and replicate count. Otherwise, differences in power cannot be attributed clearly.
Phenotype prevalence deserves particular care. In a population cohort, prevalence affects how many cases accrue for a fixed cohort size. In a case-control design with fixed numbers of recruited cases and controls, changing source-population prevalence does not by itself change that allocation, but it can affect recruitment cost, the relationship between source-population and sampled data, and the assumptions used for ascertainment. Simulate the recruitment process that will actually be used rather than treating prevalence as a universal multiplier on power.
A concise report should give the estimated power and false-positive rate for each named scenario, the number of replicates, the Monte Carlo standard errors, the rejection threshold, and the assumptions that distinguish scenarios. Report uncertainty in the biological and technical inputs separately when possible. The main value of simulation is not a single precise number; it is a transparent comparison of design choices and a clear account of which assumptions drive the result.
Check Your Understanding
Answer these questions by referring to the simulation assumptions, not just the rejection counts.
- What is the difference between estimated power under an alternative and estimated false-positive rate under a null scenario?
- Why should a null simulation include ancestry imbalance when ancestry predicts disease risk and allele frequencies vary across ancestry groups?
- If 450 of 1,000 simulated alternative studies reject, calculate estimated power and its Monte Carlo standard error.
- How could differential sequencing error between cases and controls affect a null simulation?
- Why does increasing the number of simulation replicates not resolve uncertainty about the assumed effect sizes?